Pagination With Hotwire.

Pagination With Hotwire

In the pantheon of classic user stories over the years of modern web development, pagination is certainly well represented. The first release of the long lived will_paginate gem goes all the way back to 2008 and the early years of Rails. Amazingly, newer gems like kaminari and the latest upstart pagy have also found their way into the hearts and apps of Rails developers. Due to both the needs of performance and UX there will likely always be new twists on ways to page through an ordered list of items.

Recently I had an opportunity to add some pagination to posts being rendered in a business website building app we are working on at Flagrant. Since this is a new project using the latest Rails features this seemed like a great opportunity to see how old patterns intersect with some of the new hotness of Turbo.

Turbo-Frame Infinite Scrolling

Our first pass resulted in an implementation of the “Infinite Scroll” approach to pagination. Back in the early years of will_paginate, prototype and jQuery this was a tricky feat to pull off. Google a bit and you will still find a legacy of jQuery and js plugins that attempt pull off the illusion of having an endless stream of items without the performance penalty of eager loading all of them on initial page load. These days it is simple enough to roll your own customized solution though it can still take some clever calculations as is illustrated by this excellent GoRails Tutorial using a stimulus-js controller to drive the interaction.

Fortunately, with turbo frames this now becomes dead simple thanks to a built in lazy loading feature. There are already probably lots of examples of how to achieve this but I offer a shout out to this blog post for first demonstrating the trick to me. To recap the essence of it here using the pagy gem one needs controller endpoints such as:

# app/controllers/businesses_controller.rb
def show
  @business = Business.find(params[:id])
end

# app/controllers/businesses/posts_controller.rb
def index
  @business = Business.find(params[:id])
  @pagy, @posts = pagy(@business.posts, items: 25)
  render layout: false
end

The show view for the business might then list the posts by adding the following to its template:

<!--- app/views/businesses/show.html.erb --->
<%= turbo_frame_tag "posts", src: businesses_posts_path(params[:id]) %>

combined with a template like the following:

<!--- app/views/businesses/posts/index.html.erb --->
<%= turbo_frame_tag "posts" do %>
  <% @posts.each do |post| %>
    <li class="bg-white rounded-md shadow-md p-8 mb-4">
      <h2 class="text-3xl text-clay-darker"><%= post.title %></h2>
      <p class="text-sm mb-6">Date Published: <%= post.published_at %></p>
      <p class="text-lg"><%= post.body %></p>
    </li>
  <% end %>
  <% if @pagy.next %>
    <%= turbo_frame_tag "posts", loading: :lazy, src: businesses_posts_path(@business, page: @pagy.next) do %>
      <!--- put your loading prompt and/or spinner here --->
    <% end %>
  <% end %>
<% end %>

The magic here comes from the turbo-frame html custom element that is generated by the very simple turbo_frame_tag helper. Custom elements are self contained components encapsulating html, javascript and even css defined by tags that you name and construct for yourself. Of course Turbo has defined the turbo-frame element though in theory you could even extend it further yourself.

The turbo-frame technique used here is quite simple in that when it appears on the page it will make a request using the src attribute if it is defined and then any turbo-frame returned in the response will replace the parent turbo-frame element with the matching id with its inner html.

In our case if there are more paginated posts then an additional turbo-frame element is embedded that will make the next paginated request. In this way all the posts get nested within the russian doll of turbo frames though they essentially appear as one list on the page.

The real magic though is in the loading: :lazy attribute which makes sure that the paginated request only happens once the turbo-frame element appears in the view-port. Under the covers turbo is using an IntersectionObserver which is now a more standardized solution built into browsers for determining that an element has entered the viewport. Note that without the lazy loading all the paginated requests would happen eagerly in quick succession defeating any of the performance benefits expected from an “Infinite Scroll” solution.

It is also worth noting that repeating the “posts” id is probably a bit sloppy and a cleaner approach will be shown below. Also the original bluebash blogpost example added an action: append to the turbo frame attribute but that functionality is really only available with a different custom element - turbo-stream - but we are getting ahead of ourselves…

Read More Scrolling

As cool as it can be, sometimes infinite scrolling is not what you want. In our case we have some elements at the bottom of the page that we don’t want to keep our users from easily accessing. Our design involved requesting more paginated posts with a “Read More” link so the above solution needed to be adapted to work. I actually found this variation demonstrated in a separate blog post.

Adapting the code above we get the following:

<!--- app/views/businesses/show.html.erb --->
<%= turbo_frame_tag "posts-1", src: businesses_posts_path(params[:id]) %>

<!-- combined with a template like the following:  -->
<!--- app/views/businesses/posts/index.html.erb --->
<%= turbo_frame_tag "posts-#{@pagy.page}" do %>
  <% @posts.each do |post| %>
    <li class="bg-white rounded-md shadow-md p-8 mb-4">
      <h2 class="text-3xl text-clay-darker"><%= post.title %></h2>
      <p class="text-sm mb-6">Date Published: <%= post.published_at %></p>
      <p class="text-lg"><%= post.body %></p>
    </li>
  <% end %>
  <% if @pagy.next %>
    <%= turbo_frame_tag "posts-#{@pagy.next}" do %>
      <%= link_to businesses_posts_path(@business, page: @pagy.next) do %>
        <div class="text-center btn-outline">
          Read More
        </div>
      <% end %>
    <% end %>
  <% end %>
<% end %>

We have basically the same idea here except now we have an explicit link but since it is wrapped in a turbo-frame element it is going to trigger a turbo request that will in turn return a turbo-frame with a matching id. In actual fact under the covers this is almost exactly the same solution as the infinite scroll because when the link is clicked the turbo frame_controller will literally intercept the click event of the link and take the requested url and set it to the src attribute of the containing turbo-frame element, This will in turn trigger the eager loading of the new frame of next posts. You can actually watch the magic in action if you inspect the elements in your developer tools while clicking on the link.

However, once again we are going to get a nested list of paginated posts wrapped in the telescoping turbo frames. Maybe you don’t like that unusual html structure or perhaps you find recursive behavior a bit mind bending. Or better yet, you want to learn something about the other custom element brought to you with Turbo. In that case, I have one more solution for you to consider.

Explicit Turbo Stream Pagination

This solution may not be quite as elegant but it is a bit easier to follow and understand which can be very important when working with a larger team maintaining the same code base.

While turbo-stream elements are perhaps more associated with real time updates delivered in Rails apps via websockets with ActionCable the elements themselves function just as well when delivered via standard http requests. Though similar to turbo-frames, a turbo-stream tag is a bit more flexible as it can perform 7 different dom manipulations.

In our case we will want to be able to append posts to our list and update our “Read More” button with the next pagy link. Our templates would look like this:

<!--- app/views/businesses/show.html.erb --->
<ul id="business-posts">
</ul>
<div id="read-more-link">
</div>
<%= turbo_frame_tag "post-streams", src: businesses_posts_path(@business) %>

<!--- app/views/businesses/posts/index.html.erb --->
<%= turbo_frame_tag "post-streams" do %>
  <%= turbo_stream.append("business-posts") do %>
    <% @posts.each do |post| %>
      <li class="bg-white rounded-md shadow-md p-8 mb-4">
        <h2 class="text-3xl text-clay-darker"><%= post.title %></h2>
        <p class="text-sm mb-6">Date Published: <%= post.published_at %></p>
        <p class="text-lg"><%= post.body %></p>
      </li>
    <% end %>
  <% end %>
  <% if @pagy.next %>
    <%= turbo_stream.update("read-more-link") do %>
      <%= link_to businesses_posts_path(@business, page: @pagy.next), data: { turbo-frame: "posts-streams" do %>
        <div class="text-center btn-outline">
          Read More
        </div>
      <% end %>
    <% end %>
  <% end %>
<% end %>

We are using the turbo_stream helper that comes with the turbo-rails gem but again these are pretty simple wrappers that generate basic tags that look something like:

<turbo-stream target="business-posts" action="append">
  <template>
    <li>....</li>
    ...
  </template>
</turbo-stream>

Note that the turbo-frame ids must match for all this to work because the element will try to find its matching turbo-frame when it is returned in the server response.

Another important trick here is to add the turbo-frame data-attribute to the link in order for it to correctly trigger a turbo-stream format request successfully from the server. You could just use a form button and make a POST, PUT, PATCH or DELETE request and it would just work. However this data-attribute seems to be needed for a GET request which seems like a more RESTful standard in this case since we are just requesting data and not doing any persisting or mutating.

Conclusion

So there you have it: a few more ways to handle pagination with automatic or manual appending of items with the new turbo hot(wire)ness. If some of this feels a bit like a happy return to something old and familiar, like the glory days of SJR (Server-generated JavaScript Responses) and before front end heavy single page apps started taking over, then you are not alone. Indeed, with tools like Hotwire we are able to create sophisticated and performant user experiences while keeping more of our logic and attention focused in the backend language and framework Rails developers have grown to love more and more over the years.

If you’re looking for a team to help you discover the right thing to build and help you build it, get in touch.