LokiJS Disk Storage Adapter for NeutralinoJS

One of the choices I’ve had to make in building my SamplesOrganizer app is what database to use. I wanted an in-memory database that could handle a large amount of data, and persist it to disk without adding tens or hundreds of megabytes.

Enter LokiJS. Loki is a document database (like Mongo) that simply stores your data as a flat JSON file on disk. Per its documentation it’s good and speedy for database files up to 100 megabytes. Given my average record size, it should be performant up to about 80,000 sounds, maybe more.

I usedLokiJS for a project back in 2016 when it was being actively developed. Now, in 2024, it isn’t actively developed anymore, but it reached a reasonable level of maturity and it’s simple and non-bloating, helping my keep the overall app down to a minimal size.

Unfortunately, because Neutralino isn’t running Node like Electron does, the LokiJS Node.fs-based persistence adapter for local storage doesn’t work, and based on my experiments, the LokiJS persistence adapter that uses IndexedDb runs into Apple privacy protections that delete IndexedDb storage data after a week of not using the app. Thus I had to create my own adapter.

I’ve sort of done this before

This is not the first time I’ve created an adapter to persist data between sessions. When I was an evangelist with Amazon Alexa, I created a local persistence adapter to do essentially the same thing for Alexa skill session data, a mimic of the AWS S3 persistence adapter that is built in, but to store your data locally when developing and debugging locally.

Sadly, I put it in the Alexa Cookbook repository, which I championed decommissioning before I left Alexa 3.5 years ago. It was decommissioned and archived for a while. Now it’s not available at all. Still, if you’re the type who says “pics or it didn’t happen,” I demonstrated it on a Alexa Developers India presentation (which I had to be up REALLY early on a Saturday morning to do).

The “LokiJS Data Persistence Adapter for Neutralino” Code

A couple of notes first. The data_path variable is a constant in my app that is determined outside the class. It holds the result of a utility function that gets the library path from Neutralino, adds a directory name for the app’s data, and creates the directory if it hasn’t been created. The reason for the constant is that other app data is stored in that directory, so to prevent calling that function many times, it’s called once in initialization and the constant is set.

Also UTILS.error_modal is a utility function in my code, used by multiple functions to display error messages as modal pop-ups.

Here’s the code for the function that establishes the data_path and ensures it exists.

Of course, you’d replace SamplesOrganizer with the name you want to use for your app’s data. This said, please use something that will be meaningful to the user. If they browse this folder, they shouldn’t look at your directory and wonder what app’s data it’s storing.

Now here’s the code for the adapter.

In the long run, I may swap out a more actively maintained database engine or add a plugin architecture for a BYO database option. For now, this database is functional and extremely lightweight, helping me keep my executable size under 4 megabytes in most instances (Mac Universal Binary is larger).

As I’ve said, I plan to open source the code after a private beta period to work out some of the kinks and allow me to refactor my code for easier readability/maintainability (since I get tunnel vision on an objective and coding standards can give way to “Git ‘er Done“). But this bit is, I guess, ready for release. So take it and go.