Unable to use zeromq and serialport in one package with npm as package manager

[tomasz-szypenbejl-td]

Describe the bug
npm install (or npm ci) command will hang forever in any any project that has both zeromq and serialport in the dependencies. It only happens on unix-like systems - I was able to reproduce the issue on MacOS and Linux (both running on x86_64 CPU), but not on MS Windows. Interestingly, the issue only occurs with npm. Switching to yarn or pnpm is my current workaround for the issue.

The issue seems to be caused by the custom fork of node-gyp-build.

Reproducing
The steps below involve Docker use (to provide a consistent environment in which the issue can be reproduced). If you happen to be using MacOS or Linux, and you happen to have typical development tools installed, please feel free to skip straight to point 3 (any contemporary version of node and npm should be OK).

1. Start a new instance of Debian 11 with node 18 preinstalled in Docker:
   $ docker run -it node:18-bullseye /bin/bash

2. (In the shell of the docker container just started) install cmake:
   root@42f71b8758da:/# apt update && apt install cmake -y

3. Create a temporary folder to reproduce the issue and cd into that folder:
   root@42f71b8758da:/# mkdir reproduce
root@42f71b8758da:/# cd reproduce/

4. Initialize a new package:
   root@42f71b8758da:/reproduce# npm init -y

5. Install the latest version of zeromq (6.0.0-beta.16 at the time of writing this bug report):
   root@42f71b8758da:/reproduce# npm install zeromq

6. Install the latest version of serialport (10.5.0 at the time of writing this bug report):
   root@42f71b8758da:/reproduce# npm install serialport
   So far so good, you should not be experiencing the issue yet - both zeromq and serialport should install just fine.

7. Remove the node_modules folder (so that your temporary package is in a clear state - as if just cloned from some repository), and then try to install the dependencies specified in package.json:
   root@42f71b8758da:/reproduce# rm -rf node_modules/
root@42f71b8758da:/reproduce# npm ci
   Please feel free to use npm install instead of npm ci - it will not make much difference. Unless something has changed since I wrote this bug report, the npm command will hang forever (please wait a few minutes to make sure that the command is really not going to complete its work ever). You will typically be seeing something like
   [##################] - reify:zeromq: timing reifyNode:node_modules/resolve Completed in 317ms
   while the command is hanging. If you have reached here, you have successfully reproduced the issue.

8. If you reproduced the issue, you can do some basic investigation now. Press Ctrl+Z to put the hanging npm command to background, and then display current process tree:
   root@03f82d4b7122:/reproduce# ps xjf
   You should see something like:

    0     1     1     1 pts/0     3562 Ss       0   0:00 /bin/bash
    1  2034  2034     1 pts/0     3562 Tl       0   0:02 npm ci
 2034  2045  2034     1 pts/0     3562 T        0   0:00  \_ sh -c node-gyp-build
 2045  2046  2034     1 pts/0     3562 Tl       0   0:00      \_ node /reproduce/node_modules/.bin/node-gyp-build
 2046  2062  2034     1 pts/0     3562 T        0   0:00          \_ /bin/sh -c /usr/local/bin/node
 2062  2063  2034     1 pts/0     3562 Tl       0   0:00              \_ /usr/local/bin/node
    1  3562  3562     1 pts/0     3562 R+       0   0:00 ps xjf

9. You can verify that the grand-grand-grand child process of npm ci, that all the other process are waiting for, is executing in serialport's directory:
   root@03f82d4b7122:/reproduce# ls -l /proc/2063/cwd
   Your PID number will obviously be different than the one in the example (2063). You should see the following command output:
   lrwxrwxrwx 1 root root 0 Feb 17 13:05 /proc/2063/cwd -> /reproduce/node_modules/@serialport/bindings-cpp

10. You can also check where the symlink at node_modules/.bin/node-gyp-build is pointing to:
    root@03f82d4b7122:/reproduce# ls -l /reproduce/node_modules/.bin/node-gyp-build
    You will most likely see the following output:
    lrwxrwxrwx 1 root root 32 Feb 17 13:03 /reproduce/node_modules/.bin/node-gyp-build -> ../@aminya/node-gyp-build/bin.js
    This is why I think the custom fork for node-gyp-build is guilty. It seems that it replaces the original node-gyp-build in the node_modules/.bin folder, but it does not work well for serialport.

You can try repeating the steps with a small change - in point 5 install an old version of zeromq that did not have the custom fork of node-gyp-build yet:
root@42f71b8758da:/reproduce# npm install [email protected]
Then in point 7 make sure to use npm ci command rather than npm install (as the later one could install a newer version of zeromq). With this small change everything should work fine and npm ci command should complete normally (just give it a few minutes).

Expected behavior
npm install and npm ci commands should complete normally when installing dependencies from package.json that include both zeromq and serialport.

Tested on

• OS: MacOS 12.6, MacOS 13.1, Debian 10, Debian 11 (all systems running on x86_64)
• ZeroMQ.js version: 6.0.0-beta.16
• node.js version: 16.14.2, 18.14.0
• npm version: 8.5.0, 9.3.1, 9.4.0

[SeverinAlexB]

Same here on Ubuntu,

• ZeroMQ.js 6.0.0-beta.16
• Nodejs 16, 18, and 20.
• npm v8.19.2 and v9.6.6

Nodejs 20 actually shows an error:

npm ERR! command failed
npm ERR! command sh -c -- (shx test -f ./script/build.js || run-s build.js) && cross-env npm_config_build_from_source=true node-gyp-build
npm ERR! sh: 1: cross-env: Permission denied

Regular cross-env permission workarounds do NOT work.

[RomanKisilenko]

Same issue here, this time the custom fork of node-gyp-build used by latest beta versions of zeromq 6.0.0 conflicts with "bufferutil": "~4.0.7".