Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unfork one file, and prepare for unforking more #5

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

gnprice
Copy link
Member

@gnprice gnprice commented Aug 28, 2020

This is a first step toward zulip/zulip-mobile#4242.

Most of this is one-time stuff; just the one commit "Unfork one module, fenced_code." represents what we'll repeat for each successive file.

After this file, emoji.js is ready for similar treatment. Then when more files are ready from the web side, we can act on those in turn.

/cc @showell

This whole zulip-markdown-parser package is an old fork of the
Markdown implementation in the Zulip web frontend.

We've begun making parts of that implementation -- the current
version of it, with all the new features and bugfixes of the last
several years -- available in the @zulip/shared package.  Adding
a dependency on that is the first step here toward replacing this
fork with that current version, piece by piece.
Without this, if we start using something from @zulip/shared and
then try to run Jest, it fails with a SyntaxError.

The fix is to tell Jest to have Babel compile the file first.
We can do this with the steps described here (on an issue
reporting the same error message I saw before making this fix):
  jestjs/jest#9395 (comment)
This is the first step of zulip/zulip-mobile#4242: we're taking one
module of this Markdown implementation which was forked in 2017 from
the webapp, and unforking it by switching to the current version,
provided in @zulip/shared.  We'll go on to unfork other modules one by
one until this package is a trivial wrapper for part of @zulip/shared,
and then its user (the Zulip mobile app) can switch to using the
latter directly.

Unforking fenced_code requires two updates to the rest of the code:

 * Following zulip/zulip@8e356368f, the fenced_code module no longer
   exports a function set_escape_func, and its call site can simply
   disappear.  The new version of fenced_code.js always escapes.

   (That commit made a number of other changes in other parts of the
   Markdown implementation; we'll get the benefit of those when we
   start sharing those parts too.)

 * Paralleling zulip/zulip@8e9317582, we update some test cases.

I inspected the changes within the fork, like so:

  $ git checkout 408084b  # this repo's initial commit
  $ git --git-dir=../zulip/.git diff {4f4d284:static/js/,}fenced_code.js
  $ git log --stat -p ..master fenced_code.js

and there's nothing that isn't taken care of upstream; it just adds
some `require` calls and converts some uses of underscore to lodash.

I also inspected the changes upstream:

  # in a zulip/zulip worktree
  $ git log --stat -p --follow 4f4d284.. static/shared/js/fenced_code.js

and there's nothing else there that looks like it would depend on
changes elsewhere.
I.e., ran `yarn upgrade`.

Because this only touches the yarn.lock file, it has no effect when
this library is *used* by some other project -- it only has any effect
when running local development scripts.  And there's only one of
those, namely Jest.  Tests still pass, so success.
I.e., ran `yarn upgrade --latest`.

As far as anything *using* this library is concerned, the only
change here that matters is the one to the non-dev dependency katex.
We're already using the latest katex in the webapp, so it should be
no problem here.
@gnprice gnprice requested a review from chrisbobbe August 28, 2020 06:57
@gnprice
Copy link
Member Author

gnprice commented Aug 28, 2020

(In addition to this repo's own tests, I've tested these changes manually in the mobile app, using yarn link. Once this is merged, I'll publish zulip-markdown-parser to NPM and then bump the version referred to in zulip-mobile's package.json.)

Copy link

@chrisbobbe chrisbobbe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! See one comment, below; otherwise, this looks good.

I also tested this PR by setting zulip-markdown-parser to gnprice/zulip-markdown-parser#pr-unfork-1 in package.json and running yarn, and all seems to work fine.

@@ -0,0 +1,5 @@
module.exports = {
transform: {
'.\\.js$': 'babel-jest',

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I'm still getting the following error when we start using code from @zulip/shared in the next commit:

 FAIL  __tests__/preview-tests.js
  ● Test suite failed to run

    Jest encountered an unexpected token

    This usually means that you are trying to import a file which Jest cannot parse, e.g. it's not plain JavaScript.

    By default, if Jest sees a Babel config, it will use that to transform your files, ignoring "node_modules".

    Here's what you can do:
     • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
     • If you need a custom transformation specify a "transform" option in your config.
     • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

    You'll find more details and examples of these config options in the docs:
    https://jestjs.io/docs/en/configuration.html

    Details:

    /Users/chrisbobbe/dev/zulip-markdown-parser/node_modules/@zulip/shared/js/fenced_code.js:1
    import katex from "katex";
    ^^^^^^

    SyntaxError: Cannot use import statement outside a module

      14 | const marked = require('./marked');
      15 | const hash_util = require('./hash_util');
    > 16 | const fenced_code = require('@zulip/shared/js/fenced_code.js');
         |                     ^
      17 | const emoji = require('./emoji');
      18 | var emoji_codes = require('./emoji_codes');
      19 | const katex = require('katex');

      at Runtime._execModule (node_modules/jest-runtime/build/index.js:1179:56)
      at Object.<anonymous> (markdown.js:16:21)
      at Object.<anonymous> (parseMarkdown.js:1:18)
      at Object.<anonymous> (__tests__/preview-tests.js:3:23)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It goes away when I apply the following to jest.config.js (like we do in zulip-mobile):

+ const transformModulesWhitelist = ['@zulip/shared'];
+
+ const transformIgnorePattern = `node_modules/(?!${transformModulesWhitelist.join('|')})`;
+
  module.exports = {
    transform: {
      '.\\.js$': 'babel-jest',
    },
+   transformIgnorePatterns: [transformIgnorePattern],
  };

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the first item in that error's list of possible solutions—

     • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.

—it seems like anything in node_modules won't be transformed by default.

@chrisbobbe
Copy link

Ah, I just noticed another thing, @gnprice: the author field on these commits has an email address that differs from the @zulip.com one you use for zulip-mobile (in case you wanted to change that). 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants