Synchronize compat_features with mdn/browser-compat-data tags

[ddbeck]

Goal

When we override compat_features in a YAML file here, those changes ought to propagate to mdn/browser-compat-data. Likewise, new features originating in mdn/browser-compat-data ought to bubble up to web-features.

Background

Open Web Docs is working on several tasks including:

Provide synchronization of browser-compat-data keys between the mdn/browser-compat-data and web-platform-dx/web-features repositories. Contribute source code required to synchronize keys to their respective repositories on GitHub.

(see openwebdocs/project#208)

But we're not capturing this work here on the repo anywhere. This issue is attempting cover it.

[ddbeck]

There's two sides of this: upstreaming and downstreaming.

Downstreaming

Downstreaming is sending changes from the upstream side (mdn/browser-compat-data) to the downstream (web-platform-dx/web-features).

I'd expect a web-features workflow along these lines:

1. Start by upgrading @mdn/browser-compat-data in web-features (e.g., via Dependabot, repository/workflow dispatch, or cron workflow).
2. Run npm run bcd-sync (or similar). Do these steps:
   1. If there exists a tag in BCD that matches a compat_features list, then delete the compat_features list from the file and commit.
   2. If there exists a tag in BCD that differs from the compat_features list, then annotate the .yml file’s compat_features line with the discrepancy. Note: some discrepancies will be expected/desired (e.g., BCD is “stale”); this should not fail tests or block merging.
   3. Run npm run dist and commit.
   4. Push the changes.
3. Review. If applicable, push corrective changes to the PR. Finally, approve the PR.
4. Merge the PR.
5. After merging, if there any web-features: keys that exist in BCD that do not have a corresponding feature ID, then generate an issue (or perhaps even a PR from a template) in web-features reporting a "missing" feature.

Upstreaming

Upstreaming is sending changes from the downstream side (web-platform-dx/web-features) to the upstream side (mdn/browser-compat-data).

I'd expect an mdn/browser-compat-data workflow:

1. Start by upgrading web-features in a PR (e.g., via Dependabot, repository/workflow dispatch, or cron workflow).
2. Run some script that does these steps:
   1. Compare the list of feature IDs in web-features with the list of tags in BCD.
   2. Produce a Checks report (e.g., some Markdown) that shows what tags/features will change.
   3. Run migrate-to-bcd.
   4. Push the results to the PR.
3. Review and approve the PR.
4. Merge the PR.

[ddbeck]

Today, I met with @atopal and @Elchi3 to figure out next steps for this work.

Highlights

• We're going to continue to work on the scripts and workflows needed for this, but…
• We're not going to turn on synchronization workflows until the feature backfilling is complete. This is to minimize churn and challenging changes to YAML files.
• That means we need to…
  • Complete these PRs:
    • Add script to cleanup redundant BCD overrides #2333
    • Remove redundant compat_features keys from features with comments #2345 (partially, only moving or removing the comments)
    • Add undist script #2474
  • Create new mdn/browser-compat-data PRs to:
    • Prevent authors from creating new tags (i.e., BCD uses web-features IDs as an allowlist for tags). See Allowlist web-feature tags mdn/browser-compat-data#25614
    • Apply tags to BCD keys based on compat_features lists from web-features (periodically or some other trigger, such as web-features releases)
  • Create a new PR in web-features to document the overall new BCD key workflow (see next section for an overview)
  • Close these PRs:
    • Cleanup compat_features keys #2334
    • Create action to automatically migrate web-features tags to BCD #2471

Workflow summary

After some discussion, we're expecting two typical workflows for synchronizing compat_features with BCD tags:

1. New keys are tagged in BCD directly. In this workflow, new keys in BCD are tagged from inception. This is more likely for small, incremental changes to existing features.

   For example, a new key appears in BCD (e.g., via a Collector PR). The author of that PR knew which feature the key belongs to (e.g., due to a key being renamed) and tags the key with the correct web-features:* tag. A few days later, that data is released by BCD. That triggers the next @mdn/browser-compat-data Dependabot upgrade PR, where we regenerate web-features data based on the updated tag and remove the compat_features list from the YAML file (if the list and tagged keys match). Done.

   If necessary (e.g., a tag was assigned in error), a web-features maintainer can update a compat_features list in web-features and the work syncs to BCD as in workflow 2.

2. New keys are tagged in web-features. In this workflow, web-features receives unaccounted-for keys in the features/draft folder and BCD periodically incorporates compat_features as tags. This is more likely for new features.

   For example, a new key appears in BCD (e.g., via a Collector PR). The author of that PR doesn't know or doesn't care which feature that key belongs to and does not tag it. A few days later, that data is released by BCD. That triggers the next @mdn/browser-compat-data Dependabot upgrade PR. After that PR merges, we run the "Update draft features" workflow, which makes the new keys to appear in the features/draft folder. At this point, a web-features maintainer can assign the key to an existing feature or author a new feature. A few days later, that data is released by web-features. When BCD upgrades web-features, it'll automatically apply tags to each feature according to the compat_features list.

To prevent circular changes:

• BCD contributors must never create a new tag before the corresponding ID appears in web-features.
• web-features contributors must never manually remove a compat_features list from a feature.

To ensure ongoing completeness, web-features maintainers ought to strive to keep generated drafts to an absolute minimum.