Updating my procedural macro Rust crate DeriveSql

Putting learning into practice

Image by PIRO from Pixabay

I published a crate called derive-sql in 2022, i.e over one year before this story— see this story published in September 2022. I have worked with it and on it on and off since then — mostly making it more complicated. In October 2023, I published a story about traits, generics, and macros. Alain Couniot’s comment on the story [I can’t find a way to link directly to the comment] and this story on dependency inversion in Rust made me revisit the derive-sql crate to put knowledge in practice.

This story documents my thoughts and choices when updating the crate. I am also highlighting the questions I faced. My aim is very much to induce feedback — positive and negative — to learn and improve.

The crate is available on crates.io and the source code is hosted on github.

The views/opinions expressed in this story are my own. This story relates to my personal experience and choices. The story, demonstrations, and source code are provided in the hope that they will be useful without any warranty.

I developed the derive-sql crate using procedural macros. Following initial development, I added functionalities and complexity to support API management and other items to it. This story reflect my current thinking that a crate should do one thing well, in this case: provide typed interaction with SQL databases — targeting SQLite via the rusqlite crate initially but with consideration of future expansion.

Defining a trait

I started to understand that defining a trait is the right starting point for the above purpose. The Rust Programming Language section on derive macro describes the implementation of a HelloMacro trait. Writing my previous story and Alain’s comment led me to understand why the implementation of a trait is used as an example. A trait allows a couple of things to happen: (a) users of the functionality can use the trait and the derive macro and then later change to their own implementation of the trait and (b) test can be achieved using a mock implementation of the trait — manually or automatically such as using the mockall crate.

The definition of the trait is also where things become hard for me. Defining the trait lead to early design decision: Are the trait methods to be static, use mutable or immutable references or consume the object? Is it better to define the trait methods to return a Result even if the implementation does not need it? This problematic is expressed well by Kraig McFadden: “you will be tempted to make beautiful, generic, reusable interfaces”… Using my prior iteration of the crate, I settled for the defining the following trait Sqlable :

I made the decisions to:

• Use one trait for all methods — by opposition to using multiple traits with one method each such as a trait Insertable to support the insert method, etc.
• Use reference methods — immutable when querying information and mutable when modifying information.
• Return Results for all methods with an associated type for the error type.
• Add a delete_table method — I am not sure about whether this is a good choice or not. There is no reciprocal create_table as this can/should be implemented as part of the instantiation of the object implementing the trait and/or could be done as a check in the implementation of the trait methods.
• Use an argument of associated type Self::Selector in each of the query call. This aim to provide a mechanism for the trait implementation to pass information.
• Use associated types for the type that is stored in the database Item, the type used to pass selection information Selector and lastly the error type returned Error. I was very unsure when choosing whether or not to use associated types for each of these and felt a little that I was falling in the trap of a “beautiful, generic, reusable interface”…

One realisation I made is that the above is a choice. It is likely neither perfect nor completely disastrous (I hope). But now that the choice is made and released, I will have to conform to it.

Defining a procedural macro to implement the trait for SQLite

The prior iteration of the crate used a procedural macro without an underlying trait — but the trait functionalities and methods are taken from the previous implementation. So the implementation of the procedural macro is really just a refactoring to adjust to the trait definition with a couple of consideration:

• The procedural macro name is changed from DeriveSql to DeriveSqlite.
• The procedural macro is available as an optional feature sqlite.
• As procedural macro needs to be in their own crate, the repository includes two library crates: derive_sql which contains the trait and extras\derive-sql-sqlite which contains the procedural macro that implement the trait for SQLite. The code uses path to refer to each other crates — derive_sql needs to access derive-sql-sqlite so that the procedural macro can be re-exported, and derive-sql-sqlite needs to access the trait defined in derive-sql (in dev) so that the tests can run… It works great, but… deploying to crates.io requires all (non dev) dependencies to be referred to by version, so derive-sql-sqlite is deployed first and then derive-sql with attention paid to the version referencing in the crates respective Cargo.toml. This does raise some questions over how to best manage the development process for the trait and derive macro — essentially does one consider both to be closely coupled (in which case it may make sense for one repository with both crates and having them reference each other by path, but having to do some ‘jumping through hoops’ when deploying) or consider them to be independent and reference each other by version (and live with a deployment management challenge of managing deployment until version 1 of the trait is achieved)???

Mocking

Using a trait allow the decoupling dependency and hence add the ability to mock in testing. An example of mocking is provided in the README file as an example using the mockall crate. The mocking example uses the external trait approach of mockall to demonstrate how mocking can be achieved in a client crate.

The mocking uses a local copy of the trait definition — the associated types have to be explicitly defined unfortunately (and one seem to have to use the concrete associated type, not Self::Item, in the trait definition):

mockall defines a new struct MockSqlableStruct that implements the Sqlable trait. It can be customized to test calls and responses as shown in the example below:

This example tailored for the DeriveSql crate shows in practice the benefits of using a trait.

Parting note

This exercise improved my understanding of the benefits of implementing against an interface rather than an implementation which can be achieved using traits in Rust.

On a personal level, it challenged my tendency to aim for ‘beautiful, generic’ and pivot towards an appreciation of ‘good enough’ and ability to extent, as well as my tendency to rewrite to suit my needs rather than extend what is available. I now realise that this tendency slows down feature development.