A Soft Deletion Story Continued

Last time I talked about soft deletion, I laid out a sticky situation I got myself into. That was mostly background and context to get to this point, where I can share a solution. Away we go!

All these problems got me entirely fed up with this database-view way of handling soft deletion, and I finally — this story has been compressed for your benefit, but I dealt with this for quite a while — went back to searching for Ecto packages. I’m not sure what was different from before, but this time I found a few compelling choices. There’s a paranoid for Ecto! There are also other packages that are more explicit and less automatic, like trash and soft_repo. These are nice, but they all seem to use deleted_at with a timestamp instead of a simple is_deleted flag (and at least some of them just use the presence of a value, which takes away some of the usefulness of having the timestamp there). That, combined with things like pagination and join tables and such, seems to point to them not being a great fit for this project, at least not at this time. Then I found Ecto.Rescope. And it seemed like the perfect solution for this app.

Since it’s going to be re-used on every model that gets soft-deleted, some helpers are a nice start.

defmodule SoftDelete.SoftDeleted do
  defmacro __using__(_opts) do
    quote do
      use Ecto.Rescope

      @rescope &SoftDelete.QueryHelpers.exclude_deleted_items/1
    end
  end
end

defmodule SoftDelete.QueryHelpers do
  import Ecto.Query

  def exclude_deleted_items(query) do
    query |> where([t], t.is_deleted == false)
  end
end

Then all the model needs is to use the table for its schema and add use SoftDelete.SoftDeleted. Easy peasy, lemon squeezy.

Now the query can just use the struct-update syntax and it all works. Wait, except the activities count. That takes a little more doing, but it’s still okay.

defmodule SoftDelete.PlaceActivity do
  use Ecto.Schema
  use Ecto.Rescope

  import Ecto.Query

  alias SoftDelete.Place
  alias SoftDelete.Activity

  @rescope &__MODULE__.exclude_deleted_items/1
  schema "place_activities" do
    belongs_to :place, Place
    belongs_to :activity, Activity
  end

  def exclude_deleted_items(query) do
    query
    |> join(:inner, [pa], p in ^Place.scoped(), on: pa.place_id == p.id)
    |> join(:inner, [pa], a in ^Activity.scoped(), on: pa.activity_id == a.id)
    |> subquery()
  end
end

Now we have things that work nicely. Reads (selects) work. Joins work. Aggregation works. Writes (inserts, updates) work. And it’s mostly unobtrusive, automatic, and opt-in. Something about soft_repo makes me itch, needing to replace every Repo call with SoftRepo (or change the alias lines). And with trash, I need the explicitness of adding discarded or kept to all sorts of Repo function calls. I get why people aren’t into the automatic stuff, but I want something minimal that keeps people from accidentally including deleted results in their queries.

But it’s not all automatic, is it? I’m sure you noticed the use of ^Model.scoped() up there. That’s covered in the Rescope README, and frankly, I’m not the biggest fan. But I do accept it. It’s partly an issue of the language, and partly due to a personal choice of how to use the Query API.

It also ties into Model.unscoped(), which is how to get and operate on the entirety of the table, rather than just the non-deleted rows. This comes in pretty handy in tests (you want to make sure your soft deletion works as expected, right?), but also in those places — like the admin area — where you want to access these soft-deleted records.

I’ll note that Rescope only includes scoping and isn’t tied directly to soft deletion, which means yes you have to give it the scope to exclude deleted items. And you also have to add a little something to flag the item as deleted or remove that flag. That’s another reason to add that little SoftDeleted module I showed above. In my real app, it also includes a deleted_changeset function that’s used to change the is_deleted value. And while the context modules have manually-written functions to delete or restore an item, those are simple to write and could also likely be done with a module they use.

Anyway, as you can see, this is a lot more straightforward and less problematic than dealing with the views. So I went ahead and converted the app to use this approach, which felt fantastic. We have pretty good test coverage, so that gave me a fairly nice level of confidence when I was making this change. And of course I did plenty of poking around in the interface (locally and in our QA and UAT environments) before feeling good about pushing it out to production.

I’m not going to say that this is the correct way to handle soft deletion. I mentioned at the beginning that this can be contentious, and I think the reason is that the solution to a problem comes down to the nuances and idiosyncrasies of that particular problem. We joke (“ha ha only serious” style) that the answer is always “it depends”. There are solutions that are more flexible, more robust, or more widely applicable, but everything has tradeoffs and you can’t know what’s going to be the best solution without knowing the particulars of the problem.

So what does that mean? The database-view solution likely works in some cases, but it wasn’t what worked for me here. The packages like paranoid or trash or soft_repo would work in some cases, but it wasn’t what worked for me here. I found and built something that did work for me in the rescope package and some wrapping around it, but it won’t work in all cases.

That wraps it up for this hard problem of soft deletion. Go forth and enjoy finding your own special solutions to your idiosyncratic problems.

If you’re looking for a team to help you discover the right thing to build and help you build it, get in touch.