Persist nedb to disk in Electron renderer process (Webpack/Electron/nedb configuration problem)

Question

Problem

I'm trying to use a pure-JS database called nedb in an Electron renderer process. It uses the browser field in its package.json to swap in a browser-based storage system. This is causing my database to not actually be persisted to file.

Background

I'm using Next.js as my view framework, and its Webpack is configured for "target": "electron-renderer" for the rendering thread. Which apparently causes Webpack to process those browser directives, even though renderer processes should have access to both browser and Node APIs. This behavior isn't really documented, so I don't know how to override it.

What I have tried

I have confirmed that if I manually edit out the browser field on the local copy of node_modules/nedb/package.json, the problem goes away.

As a temporary workaround, I've pointed to my own fork of nedb that does this. But this is pretty unsatisfactory.

Other research

Curiously, this doesn't seem to be an issue for electron-vue, whose docs explicitly demonstrate use of nedb from a renderer process. That framework does, indeed, appear to use "target": "electron-renderer" in its Webpack config.

Is there a solution to this problem, perhaps by Webpack configuration?

Answer 1

You don't need to run the database on the renderer process, instead of that you could run other database that you would like, like sql, sqlite, mongodb, etc, on main process.

If you don't mind switching database, here is how you could achieve this. In Electron exist a class callled ipcMain and ipcRenderer, this classes are used to make renderer process and main process comunicate. You can send/receive any type of data with ipc.

Here is an example:

Renderer.js

const btnSave = document.getElementById('btn-save')

// Get any data from forms, etc


btn.addEventListener('click', () => {
     // ipcRender sends the data via 'receive-data-to-save-in-database' channel, you
     // you can send any type of data, and have has many args you want. In this case I 
     // sent a a empty object
     ipcRenderer.send('receive-data-to-save-in-database', {})              
})

Main.js

// ipcMain listens to channel 'receive-data-to-save-in-database'
ipcMain.on('receive-data-to-save-in-database', (event, args) => {
    // Code to save in database
    // The empty object will be received in args parameter
}) 

It's not what you wanted, but is a workaround.

For more information, I suggest you go:

ipcRenderer Docs ipcMain Docs

Answer 2

As you stated in your question and per this Github issue on the nedb package the root cause of your issue is that webpack's file resolution process reads the package.browser key in order to alias specific file paths to a different location when the target build is browser or some other value that will cause it to inspect the package.browser property.

electron-vue appears to sidestep the webpack bundling issue by treating all NPM dependencies as externals so that they don't get pulled into the application bundle and instead are expected to be defined on global by some other means. You could similarly designate nedb as an external in your webpack config and pull the Node version into your application via script tag or defining a reference to it on global some other way.

Another solution would be to create a webpack resolver plugin to override how the problematic requires for "./lib/customUtils.js" and "./lib/storage.js" get resolved, bypassing the resolution step that inspects package.browser for aliases for those file paths.

See the webpack documentation for how to pass a custom resolver plugin in your Webpack config. See the wepback/enhanced-resolve documentation for additional details on how plugins are defined and how they work.

Essentially, a plugin is an object with an apply method that takes a resolver instance and performs some step of the file resolution process. In the example below, we test to see whether the current file being resolved is in the nedb package and whether it's one of the two problematic browser aliases. If so, we exit the resolution process with the correct paths to the files. Otherwise we do nothing and defer to the normal resolution process.

// Prevents nedb from substituting browser storage when running from the
// Electron renderer thread.
const fixNedbForElectronRenderer = {
  apply(resolver) {
    resolver
      // Plug in after the description file (package.json) has been
      // identified for the import, which makes sure we're not getting
      // mixed up with a different package.
      .getHook("beforeDescribed-relative")
      .tapAsync(
        "FixNedbForElectronRenderer",
        (request, resolveContext, callback) => {
          // When a require/import matches the target files, we
          // short-circuit the Webpack resolution process by calling the
          // callback with the finalized request object -- meaning that
          // the `path` is pointing at the file that should be imported.
          const isNedbImport = request.descriptionFileData["name"] === "nedb"

          if (isNedbImport && /storage(\.js)?/.test(request.path)) {
            const newRequest = Object.assign({}, request, {
              path: resolver.join(
                request.descriptionFileRoot,
                "lib/storage.js"
              )
            })
            callback(null, newRequest)
          } else if (
            isNedbImport &&
            /customUtils(\.js)?/.test(request.path)
          ) {
            const newRequest = Object.assign({}, request, {
              path: resolver.join(
                request.descriptionFileRoot,
                "lib/customUtils.js"
              )
            })
            callback(null, newRequest)
          } else {
            // Calling `callback` with no parameters proceeds with the
            // normal resolution process.
            return callback()
          }
        }
      )
  }
}

// Register the resolver plugin in the webpack config
const config = {
  resolve: {
    plugins: [fixNedbForElectronRenderer]
  }
}