Clojure Database Migrations
I've been working on Clojure application (unfortunately it's for a friend and not public so I can't share the codebase) and as part of it I've been looking into database migrations.
There are two types of data it will store. The first is user information and various things around that. I haven't got to setting any of that up so I won't talk about that. I'll look at how to handle those migrations when I start working on that side of things. The other type of data that the database will store is relatively static. The application is never going to change the data, only lookup values that are stored. My idea is to have Snap-CI (the CI/CD server that I'm using) build this database from scratch using some SQL scripts every-time the application is pushed into production.
This process to build the database will be slower but it simplifies the process. We're only talking about 10,000s of rows so it shouldn't be too slow. I will capture some metrics to make sure this doesn't become an issue. What it means is in the future when this data needs to get updated I won't need to worry about writing a single delete statement, or an update statement. I can simply removed any insert statements that aren't required, change any insert statements that need to change and add new insert statements. It also means that I can start writing basic smoke tests around the production database.
There are two tools that I've using to achieve this:
The first is Korma. I'm not all that familiar with this at the moment but it provides a DSL so that I don't need to write raw SQL. It also supports me writing raw SQL if I so desire. This means I can use it in my production code for all of the lookups I need. I can then also use it to execute a SQL file that will create and populate a table.
The question you would probably ask is -if you're using Korma to create the tables how are you executing that? This leads to the second tool - lein-exec. Lein-exec is a Leiningen plugin that allows you to write Clojure as scripts. This means I can write scripts to create the database without needing to define the database connection more than once. I use the command line option to execute the script in the context of the project then I can simply 'require' the namespace in which I defined the database configuration.
There are two types of data it will store. The first is user information and various things around that. I haven't got to setting any of that up so I won't talk about that. I'll look at how to handle those migrations when I start working on that side of things. The other type of data that the database will store is relatively static. The application is never going to change the data, only lookup values that are stored. My idea is to have Snap-CI (the CI/CD server that I'm using) build this database from scratch using some SQL scripts every-time the application is pushed into production.
This process to build the database will be slower but it simplifies the process. We're only talking about 10,000s of rows so it shouldn't be too slow. I will capture some metrics to make sure this doesn't become an issue. What it means is in the future when this data needs to get updated I won't need to worry about writing a single delete statement, or an update statement. I can simply removed any insert statements that aren't required, change any insert statements that need to change and add new insert statements. It also means that I can start writing basic smoke tests around the production database.
There are two tools that I've using to achieve this:
The first is Korma. I'm not all that familiar with this at the moment but it provides a DSL so that I don't need to write raw SQL. It also supports me writing raw SQL if I so desire. This means I can use it in my production code for all of the lookups I need. I can then also use it to execute a SQL file that will create and populate a table.
The question you would probably ask is -if you're using Korma to create the tables how are you executing that? This leads to the second tool - lein-exec. Lein-exec is a Leiningen plugin that allows you to write Clojure as scripts. This means I can write scripts to create the database without needing to define the database connection more than once. I use the command line option to execute the script in the context of the project then I can simply 'require' the namespace in which I defined the database configuration.
Comments
Post a Comment