Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sled agent PUT /datasets should validate its constraint that (pool, kind) is unique #7311

Open
davepacheco opened this issue Jan 6, 2025 · 2 comments

Comments

@davepacheco
Copy link
Collaborator

We recently realized that because datasets never get deleted today, we can never put two datasets of a particular kind on the same zpool, even if one of those belonged to a zone that's now expunged. I wanted to test this out and see what happens. Unfortunately the details don't fit in a GitHub issue description. The short version is:

  • when adding a zone with a durable dataset, the planner can try to put the durable dataset on the same pool where an expunged zone's durable dataset previously was
  • this fails at execution-time, with sled agent noticing the id of the dataset is wrong (thankfully), but this is only after it's committed the new dataset config to its ledger

I'm going to file a separate issue for the planner here. This issue covers having sled agent validate this constraint before accepting the request and committing the ledger. I'll comment below with details on how I tested this and how it went wrong.

@davepacheco
Copy link
Collaborator Author

This was on the same a4x2 deployment where I've been testing datasets, most recently described in #7304.

My goal was to cause Reconfigurator to try to put an external DNS zone onto the same pool where an expunged external DNS zone was previously. I first ran into #7305. I pulled in #7307 and tried again. So at the starting point, I had this:

root@oxz_switch:~# omdb nexus blueprints show 4d07b7da-112f-4332-929e-4f2e483d32cc
note: Nexus URL not specified.  Will pick one from DNS.
note: using DNS server for subnet fd00:1122:3344::/48
note: (if this is not right, use --dns-server to specify an alternate DNS server)
note: using Nexus URL http://[fd00:1122:3344:101::6]:12221
blueprint  4d07b7da-112f-4332-929e-4f2e483d32cc
parent:    edac838d-b454-48b7-b4e8-0bd433fecc6c

  sled: 09301471-54be-465e-a674-45da58c0c59b (active)

    physical disks at generation 2:
    -------------------------------------------------------------
    vendor             model                serial               
    -------------------------------------------------------------
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g3_0
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g3_1
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g3_2
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g3_3
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g3_4


    omicron zones at generation 5:
    --------------------------------------------------------------------------------------------
    zone type         zone id                                disposition   underlay IP          
    --------------------------------------------------------------------------------------------
    clickhouse        0208b07b-4ccd-4e8b-b334-1ec8ecc4bae7   in service    fd00:1122:3344:103::6
    cockroach_db      67989451-63b3-48c2-aef8-c6e511fb122f   in service    fd00:1122:3344:103::3
    crucible          79da9a6d-9db8-4fd9-b2dc-ba0d2fed0bdd   in service    fd00:1122:3344:103::8
    crucible          7c5ae2a6-95a7-4cd9-8631-68477b614d97   in service    fd00:1122:3344:103::a
    crucible          a0e1f460-0a7c-4788-b4a2-3ecc2ac909ad   in service    fd00:1122:3344:103::c
    crucible          b16765f7-de2f-44c2-b6ae-4fb60f5c4d53   in service    fd00:1122:3344:103::b
    crucible          f1497d76-dc9d-40ac-9670-3b17e1a64b84   in service    fd00:1122:3344:103::9
    crucible_pantry   31170c27-6736-418a-b496-b740a5668e4c   in service    fd00:1122:3344:103::7
    external_dns      8ea2a399-24f4-44c1-a31c-9b83750de6c1   in service    fd00:1122:3344:103::4
    internal_dns      0c752586-eb3b-4857-aa35-85a1206b6efb   in service    fd00:1122:3344:3::1  
    internal_ntp      d2d54e36-5a56-491e-8910-8a2cbe10cce9   in service    fd00:1122:3344:103::d
    nexus             bdf0578f-d601-44e9-9318-5b5083d3e57d   in service    fd00:1122:3344:103::5



  sled: a21d85f9-ff13-4800-9e86-5eb7b9e62eb5 (active)

    physical disks at generation 2:
    -------------------------------------------------------------
    vendor             model                serial               
    -------------------------------------------------------------
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g0_0
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g0_1
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g0_2
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g0_3
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g0_4


    omicron zones at generation 7:
    ---------------------------------------------------------------------------------------------
    zone type         zone id                                disposition   underlay IP           
    ---------------------------------------------------------------------------------------------
    boundary_ntp      2b3c2cf8-bf97-4a7c-9327-712f1d589c7b   in service    fd00:1122:3344:101::d 
    cockroach_db      792489e0-9d08-4d21-9caa-0440b12d6ade   in service    fd00:1122:3344:101::4 
    cockroach_db      83e82c9d-e587-4a68-a18d-128712e2741f   in service    fd00:1122:3344:101::3 
    crucible          049d9f96-6e06-43a0-a924-35146efd7b8c   in service    fd00:1122:3344:101::a 
    crucible          6d03d1db-c605-4a00-9717-3bf867969ad9   in service    fd00:1122:3344:101::9 
    crucible          8ab366b1-b6f7-4467-8468-36745617618c   in service    fd00:1122:3344:101::c 
    crucible          b26d1945-d24b-41f9-b80f-77af3638941a   in service    fd00:1122:3344:101::8 
    crucible          b67c6455-b415-4404-b997-588d39ae4a6c   in service    fd00:1122:3344:101::b 
    crucible_pantry   8b81d199-146c-4ccf-aa23-f57140a6b582   in service    fd00:1122:3344:101::7 
    external_dns      13f9e4ae-4f94-4718-af29-4a5da01bac2f   in service    fd00:1122:3344:101::21
    external_dns      5c88569a-d13a-483c-b276-90730292d35b   expunged      fd00:1122:3344:101::5 
    internal_dns      c3e14e81-0d5a-4461-80f5-1144ccdf4da5   in service    fd00:1122:3344:1::1   
    nexus             b8da49f4-5b11-4773-be0c-8cbae8f31400   in service    fd00:1122:3344:101::6 



  sled: f0b13ea8-c304-4a03-ab7c-da1fb2d0c612 (active)

    physical disks at generation 2:
    -------------------------------------------------------------
    vendor             model                serial               
    -------------------------------------------------------------
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g1_0
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g1_1
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g1_2
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g1_3
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g1_4


    omicron zones at generation 5:
    --------------------------------------------------------------------------------------------
    zone type         zone id                                disposition   underlay IP          
    --------------------------------------------------------------------------------------------
    boundary_ntp      f4bcdbb7-c64b-4473-9346-e5c6c411e76d   in service    fd00:1122:3344:102::d
    cockroach_db      2c00cc83-0211-4662-9f95-ecbf1c8fc26b   in service    fd00:1122:3344:102::3
    cockroach_db      2eb1e0f2-72a3-48bc-a76e-cb5b8838938f   in service    fd00:1122:3344:102::4
    crucible          0a1d8035-c373-4de7-8d70-ebe0a4b16758   in service    fd00:1122:3344:102::c
    crucible          476d5bde-b360-4c49-8093-5d56be92c63f   in service    fd00:1122:3344:102::a
    crucible          74d4206f-6bdb-4c47-b01c-9e2a876d622e   in service    fd00:1122:3344:102::9
    crucible          a3cb63c4-0044-4705-a96f-87d83b09ea5d   in service    fd00:1122:3344:102::b
    crucible          c673bafa-fcd2-40c4-a7f9-20906391ba42   in service    fd00:1122:3344:102::8
    crucible_pantry   47d0ca61-9861-4d61-948c-890c5f4ac5d5   in service    fd00:1122:3344:102::7
    internal_dns      23d6123b-1b33-4ea9-8eb0-62607cc29679   in service    fd00:1122:3344:2::1  
    nexus             7b9677c8-0daf-4ab1-bc1d-9d655b964b33   in service    fd00:1122:3344:102::5
    oximeter          f4806e03-9ecc-49ef-8ee0-ff7aee4dab23   in service    fd00:1122:3344:102::6


 COCKROACHDB SETTINGS:
    state fingerprint:::::::::::::::::   d4d87aa2ad877a4cc2fddd0573952362739110de
    cluster.preserve_downgrade_option:   "22.1"

 METADATA:
    created by:::::::::::   b8da49f4-5b11-4773-be0c-8cbae8f31400
    created at:::::::::::   2025-01-03T03:54:27.496Z
    comment::::::::::::::   (none)
    internal DNS version:   2
    external DNS version:   2

Of note are the external DNS zones:

  • there's one on sled 09301471-54be-465e-a674-45da58c0c59b (not changing as part of this)
  • there's an in-service zone with id 13f9e4ae-4f94-4718-af29-4a5da01bac2f on a21d85f9-ff13-4800-9e86-5eb7b9e62eb5. The output doesn't show this (omdb nexus blueprints show could show datasets #7303) but this one's durable dataset's pool is oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679.
  • there's an expunged one with id 5c88569a-d13a-483c-b276-90730292d35b, also on a21d85f9-ff13-4800-9e86-5eb7b9e62eb5. Its durable dataset's pool is oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc.

This is too few zones, so the next attempt to generate a blueprint would add a new external-dns zone. I want it to land on a disk with an expunged external-dns zone, so I'm first going to expunge zone 13f9e4ae-4f94-4718-af29-4a5da01bac2f. This is what led to #7305, but now I'm trying with the fix for that:

root@oxz_switch:~# ./reconfigurator-cli
generated RNG seed: impossibly-together-candlefish
〉load reconfigurator-2.out de0ce350-7ab7-4b57-8104-d085b82d00ac
loaded data from "reconfigurator-2.out"
result:
  system:
    using collection de0ce350-7ab7-4b57-8104-d085b82d00ac as source of sled inventory data
    loaded sleds: 09301471-54be-465e-a674-45da58c0c59b, a21d85f9-ff13-4800-9e86-5eb7b9e62eb5, f0b13ea8-c304-4a03-ab7c-da1fb2d0c612
    loaded collections: 7253276d-595a-437a-b611-3b98c9ed107a, 673127c7-f791-4498-a345-bc307bc914bb, bf2dc06d-2ec3-4938-a3a8-01f66cc5710f, de0ce350-7ab7-4b57-8104-d085b82d00ac
    loaded blueprints: 2e24acb1-21c8-469a-ace5-bec12f89a539, 4d07b7da-112f-4332-929e-4f2e483d32cc, 831679c9-26f8-4e3b-9873-e2522cfdc087, edac838d-b454-48b7-b4e8-0bd433fecc6c, f55291a6-4265-4178-a269-7df22eb0a2b7
    loaded service IP pool ranges: [V4(Ipv4Range { first: 198.51.100.20, last: 198.51.100.29 })]
    loaded internal DNS generations: (none)
    loaded external DNS generations: (none)
  config:
    configured external DNS zone name: oxide.test
    configured silo names: default-silo, recovery

〉blueprint-edit 4d07b7da-112f-4332-929e-4f2e483d32cc expunge-zone 13f9e4ae-4f94-4718-af29-4a5da01bac2f
blueprint 296265f5-ed95-48e4-9eb2-8860c7c006c7 created from blueprint 4d07b7da-112f-4332-929e-4f2e483d32cc: expunged zone 13f9e4ae-4f94-4718-af29-4a5da01bac2f from sled a21d85f9-ff13-4800-9e86-5eb7b9e62eb5
〉blueprint-save 296265f5-ed95-48e4-9eb2-8860c7c006c7 blueprint-external-dns-2-expunged.json
saved blueprint 296265f5-ed95-48e4-9eb2-8860c7c006c7 to "blueprint-external-dns-2-expunged.json"
〉


〉blueprint-diff 4d07b7da-112f-4332-929e-4f2e483d32cc 296265f5-ed95-48e4-9eb2-8860c7c006c7
from: blueprint 4d07b7da-112f-4332-929e-4f2e483d32cc
to:   blueprint 296265f5-ed95-48e4-9eb2-8860c7c006c7

 UNCHANGED SLEDS:

  sled 09301471-54be-465e-a674-45da58c0c59b (active):

    physical disks at generation 2:
    -------------------------------------------------------------
    vendor             model                serial
    -------------------------------------------------------------
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g3_0
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g3_1
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g3_2
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g3_3
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g3_4


    datasets at generation 2:
    -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    dataset name                                                                                                   dataset uuid                           quota     reservation   compression
    -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/cockroachdb                                                     355ddd2b-b336-45c6-903f-ecdc99b26289   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crucible                                                              36b554b7-d442-4c58-96fe-71f88d9145e6   none      none          off
    oxp_29f71bd8-8d07-44f9-a587-8dd6552d6b3d/crucible                                                              53b65adf-99c3-472b-849c-9385e61c7f80   none      none          off
    oxp_639dcdf6-d036-4e59-94d8-9a05706f2a6e/crucible                                                              ee41d385-8bc3-4932-971a-2f2b1b3461a1   none      none          off
    oxp_b5c23cec-84ac-4810-9077-3f960af72fcd/crucible                                                              ab04e945-3a16-4a33-8e8b-a87b7d51df6f   none      none          off
    oxp_b9dc2f06-bf04-463c-8ffe-79d8357d852b/crucible                                                              9f46899f-ee8c-46e1-b6a3-99f74974379f   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/clickhouse                                                      ff1f90df-76b9-46a1-b2aa-c960db481817   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/external_dns                                                    5c81c9ca-43b3-43a2-ae34-2b2d09544492   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/internal_dns                                                    c2df2f3e-9b96-48ec-bcf2-23ffccbc3c05   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/zone                                                            0e6b946e-f23d-402d-87d0-6531ef26c24a   none      none          off
    oxp_29f71bd8-8d07-44f9-a587-8dd6552d6b3d/crypt/zone                                                            48566631-3eba-4cd5-b4e9-48bb9657d8de   none      none          off
    oxp_639dcdf6-d036-4e59-94d8-9a05706f2a6e/crypt/zone                                                            644e63c6-ff9b-4ac0-a932-b1063b2a0a02   none      none          off
    oxp_b5c23cec-84ac-4810-9077-3f960af72fcd/crypt/zone                                                            c04b17c8-5626-409a-8eb1-f0b797fe9788   none      none          off
    oxp_b9dc2f06-bf04-463c-8ffe-79d8357d852b/crypt/zone                                                            f760e7e6-089d-4743-a339-1e0d882a40aa   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/zone/oxz_clickhouse_0208b07b-4ccd-4e8b-b334-1ec8ecc4bae7        d343db99-4aa1-4f6c-aa53-7be34e509d36   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/zone/oxz_cockroachdb_67989451-63b3-48c2-aef8-c6e511fb122f       9c747a15-0ce0-41bd-bc2f-d7ee792a3592   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/zone/oxz_crucible_79da9a6d-9db8-4fd9-b2dc-ba0d2fed0bdd          6ec06bb2-9fd0-45d0-a35b-d6cbf894537d   none      none          off
    oxp_639dcdf6-d036-4e59-94d8-9a05706f2a6e/crypt/zone/oxz_crucible_7c5ae2a6-95a7-4cd9-8631-68477b614d97          a4dd3c84-8616-4182-8214-c9677a57318e   none      none          off
    oxp_29f71bd8-8d07-44f9-a587-8dd6552d6b3d/crypt/zone/oxz_crucible_a0e1f460-0a7c-4788-b4a2-3ecc2ac909ad          a93a6377-6d78-4c34-b220-f2c4926e4795   none      none          off
    oxp_b5c23cec-84ac-4810-9077-3f960af72fcd/crypt/zone/oxz_crucible_b16765f7-de2f-44c2-b6ae-4fb60f5c4d53          660a8beb-683a-4c00-82e9-b8f19e9f86fe   none      none          off
    oxp_b9dc2f06-bf04-463c-8ffe-79d8357d852b/crypt/zone/oxz_crucible_f1497d76-dc9d-40ac-9670-3b17e1a64b84          37896114-bb11-402a-a1a2-f0fe800d794d   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/zone/oxz_crucible_pantry_31170c27-6736-418a-b496-b740a5668e4c   7fc29a07-ce43-41d9-9a1e-1bfd3be35a46   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/zone/oxz_external_dns_8ea2a399-24f4-44c1-a31c-9b83750de6c1      defada08-7725-4d45-acec-2fe308bae2e7   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/zone/oxz_internal_dns_0c752586-eb3b-4857-aa35-85a1206b6efb      28d664b6-8003-489b-b824-262025837f56   none      none          off
    oxp_29f71bd8-8d07-44f9-a587-8dd6552d6b3d/crypt/zone/oxz_nexus_bdf0578f-d601-44e9-9318-5b5083d3e57d             30179c95-0623-468d-8b25-b78499725884   none      none          off
    oxp_639dcdf6-d036-4e59-94d8-9a05706f2a6e/crypt/zone/oxz_ntp_d2d54e36-5a56-491e-8910-8a2cbe10cce9               c6649371-7506-4fa3-ad41-f4d7666bc1a3   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/debug                                                           50e80495-eb69-4716-8fa4-9f697b454498   100 GiB   none          gzip-9
    oxp_29f71bd8-8d07-44f9-a587-8dd6552d6b3d/crypt/debug                                                           f8a339ec-c736-44cd-b562-d845da329ece   100 GiB   none          gzip-9
    oxp_639dcdf6-d036-4e59-94d8-9a05706f2a6e/crypt/debug                                                           7f64c446-bc62-4885-8d74-7a40906c6f20   100 GiB   none          gzip-9
    oxp_b5c23cec-84ac-4810-9077-3f960af72fcd/crypt/debug                                                           59d97482-de4f-4f5a-9961-d2b00126a6a6   100 GiB   none          gzip-9
    oxp_b9dc2f06-bf04-463c-8ffe-79d8357d852b/crypt/debug                                                           914cc29f-b2cd-4128-9393-18e8d6715730   100 GiB   none          gzip-9


    omicron zones at generation 5:
    --------------------------------------------------------------------------------------------
    zone type         zone id                                disposition   underlay IP
    --------------------------------------------------------------------------------------------
    clickhouse        0208b07b-4ccd-4e8b-b334-1ec8ecc4bae7   in service    fd00:1122:3344:103::6
    cockroach_db      67989451-63b3-48c2-aef8-c6e511fb122f   in service    fd00:1122:3344:103::3
    crucible          79da9a6d-9db8-4fd9-b2dc-ba0d2fed0bdd   in service    fd00:1122:3344:103::8
    crucible          7c5ae2a6-95a7-4cd9-8631-68477b614d97   in service    fd00:1122:3344:103::a
    crucible          a0e1f460-0a7c-4788-b4a2-3ecc2ac909ad   in service    fd00:1122:3344:103::c
    crucible          b16765f7-de2f-44c2-b6ae-4fb60f5c4d53   in service    fd00:1122:3344:103::b
    crucible          f1497d76-dc9d-40ac-9670-3b17e1a64b84   in service    fd00:1122:3344:103::9
    crucible_pantry   31170c27-6736-418a-b496-b740a5668e4c   in service    fd00:1122:3344:103::7
    external_dns      8ea2a399-24f4-44c1-a31c-9b83750de6c1   in service    fd00:1122:3344:103::4
    internal_dns      0c752586-eb3b-4857-aa35-85a1206b6efb   in service    fd00:1122:3344:3::1
    internal_ntp      d2d54e36-5a56-491e-8910-8a2cbe10cce9   in service    fd00:1122:3344:103::d
    nexus             bdf0578f-d601-44e9-9318-5b5083d3e57d   in service    fd00:1122:3344:103::5


  sled f0b13ea8-c304-4a03-ab7c-da1fb2d0c612 (active):

    physical disks at generation 2:
    -------------------------------------------------------------
    vendor             model                serial
    -------------------------------------------------------------
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g1_0
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g1_1
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g1_2
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g1_3
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g1_4


    datasets at generation 2:
    -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    dataset name                                                                                                   dataset uuid                           quota     reservation   compression
    -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/cockroachdb                                                     2cfac653-9dba-492a-b4e5-c8efe5a1e1b9   none      none          off
    oxp_9d453deb-a057-438f-b555-627c501d9257/crypt/cockroachdb                                                     12feda67-cf20-4f01-8094-69a956526990   none      none          off
    oxp_60155e33-c180-4490-9f28-0cb4003ede47/crucible                                                              d597754a-6fe3-4d18-b8be-8dc0a269628d   none      none          off
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crucible                                                              beee4d0a-44b9-4298-9014-da523dd9eaa7   none      none          off
    oxp_9d453deb-a057-438f-b555-627c501d9257/crucible                                                              644161d1-8a5f-4609-ae2d-21c4ae54fdd4   none      none          off
    oxp_c1b87796-8265-4c6b-b41a-be7de82934f6/crucible                                                              5d8edd26-0cd7-434e-af7b-c50a319cd1ea   none      none          off
    oxp_f63bc936-be56-4092-a2cb-6a8508f8a23d/crucible                                                              cdf87aea-8f8f-4e49-b921-db848d6e25ef   none      none          off
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/internal_dns                                                    b1e1c105-638d-4401-a5af-534f219a11f4   none      none          off
    oxp_60155e33-c180-4490-9f28-0cb4003ede47/crypt/zone                                                            d621f634-68d8-4741-85bd-53e82095c094   none      none          off
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/zone                                                            7c00d8d6-86c6-4446-a90f-24c62ac741c4   none      none          off
    oxp_9d453deb-a057-438f-b555-627c501d9257/crypt/zone                                                            b5fcfc17-2954-484a-bbf8-8c51b97a7aa3   none      none          off
    oxp_c1b87796-8265-4c6b-b41a-be7de82934f6/crypt/zone                                                            aacb001f-2c30-427d-96cf-10f553de5446   none      none          off
    oxp_f63bc936-be56-4092-a2cb-6a8508f8a23d/crypt/zone                                                            662d12c6-b68b-4aac-b9be-9c999bf5b05e   none      none          off
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/zone/oxz_cockroachdb_2c00cc83-0211-4662-9f95-ecbf1c8fc26b       1478643a-989f-43e3-bc24-4ab2c9f40d15   none      none          off
    oxp_9d453deb-a057-438f-b555-627c501d9257/crypt/zone/oxz_cockroachdb_2eb1e0f2-72a3-48bc-a76e-cb5b8838938f       905d06de-7198-41a7-a696-35d1c66bc882   none      none          off
    oxp_60155e33-c180-4490-9f28-0cb4003ede47/crypt/zone/oxz_crucible_0a1d8035-c373-4de7-8d70-ebe0a4b16758          ac0ca38c-128a-42a5-b87a-670d7b807ed4   none      none          off
    oxp_f63bc936-be56-4092-a2cb-6a8508f8a23d/crypt/zone/oxz_crucible_476d5bde-b360-4c49-8093-5d56be92c63f          d642b0d0-1f61-4ad0-9840-98464d3b4ba2   none      none          off
    oxp_9d453deb-a057-438f-b555-627c501d9257/crypt/zone/oxz_crucible_74d4206f-6bdb-4c47-b01c-9e2a876d622e          19a81e6a-938a-4579-bf95-e1105fe1b0a5   none      none          off
    oxp_c1b87796-8265-4c6b-b41a-be7de82934f6/crypt/zone/oxz_crucible_a3cb63c4-0044-4705-a96f-87d83b09ea5d          7df8ef54-cb4a-4676-b163-7d0418043127   none      none          off
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/zone/oxz_crucible_c673bafa-fcd2-40c4-a7f9-20906391ba42          88a432f9-1f5f-4799-a303-ab5806f73f3f   none      none          off
    oxp_60155e33-c180-4490-9f28-0cb4003ede47/crypt/zone/oxz_crucible_pantry_47d0ca61-9861-4d61-948c-890c5f4ac5d5   3cbb1ed7-3290-428d-bebb-f54297c09909   none      none          off
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/zone/oxz_internal_dns_23d6123b-1b33-4ea9-8eb0-62607cc29679      74b11e9a-bf6a-458d-a2c7-740576494cc9   none      none          off
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/zone/oxz_nexus_7b9677c8-0daf-4ab1-bc1d-9d655b964b33             0d21ae90-6ee4-4422-9bc6-f336fd4b9ff5   none      none          off
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/zone/oxz_ntp_f4bcdbb7-c64b-4473-9346-e5c6c411e76d               7a9e077f-5512-4f8c-b780-36c326aa0ad5   none      none          off
    oxp_60155e33-c180-4490-9f28-0cb4003ede47/crypt/zone/oxz_oximeter_f4806e03-9ecc-49ef-8ee0-ff7aee4dab23          e48d0b5a-2a9e-4794-b45c-858bf8ebf833   none      none          off
    oxp_60155e33-c180-4490-9f28-0cb4003ede47/crypt/debug                                                           5f96f85f-15df-4b76-bc99-7cdd1e07ff16   100 GiB   none          gzip-9
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/debug                                                           5aa39aed-b571-4c7d-b8ae-040c64cfe25d   100 GiB   none          gzip-9
    oxp_9d453deb-a057-438f-b555-627c501d9257/crypt/debug                                                           45fcfb99-bc89-4c87-b5c6-ee0320d3420c   100 GiB   none          gzip-9
    oxp_c1b87796-8265-4c6b-b41a-be7de82934f6/crypt/debug                                                           d8ffad03-2ea6-432f-beb2-a5b2fc4bf885   100 GiB   none          gzip-9
    oxp_f63bc936-be56-4092-a2cb-6a8508f8a23d/crypt/debug                                                           9156026c-38ed-4ee5-b49f-b4682b7a2702   100 GiB   none          gzip-9


    omicron zones at generation 5:
    --------------------------------------------------------------------------------------------
    zone type         zone id                                disposition   underlay IP
    --------------------------------------------------------------------------------------------
    boundary_ntp      f4bcdbb7-c64b-4473-9346-e5c6c411e76d   in service    fd00:1122:3344:102::d
    cockroach_db      2c00cc83-0211-4662-9f95-ecbf1c8fc26b   in service    fd00:1122:3344:102::3
    cockroach_db      2eb1e0f2-72a3-48bc-a76e-cb5b8838938f   in service    fd00:1122:3344:102::4
    crucible          0a1d8035-c373-4de7-8d70-ebe0a4b16758   in service    fd00:1122:3344:102::c
    crucible          476d5bde-b360-4c49-8093-5d56be92c63f   in service    fd00:1122:3344:102::a
    crucible          74d4206f-6bdb-4c47-b01c-9e2a876d622e   in service    fd00:1122:3344:102::9
    crucible          a3cb63c4-0044-4705-a96f-87d83b09ea5d   in service    fd00:1122:3344:102::b
    crucible          c673bafa-fcd2-40c4-a7f9-20906391ba42   in service    fd00:1122:3344:102::8
    crucible_pantry   47d0ca61-9861-4d61-948c-890c5f4ac5d5   in service    fd00:1122:3344:102::7
    internal_dns      23d6123b-1b33-4ea9-8eb0-62607cc29679   in service    fd00:1122:3344:2::1
    nexus             7b9677c8-0daf-4ab1-bc1d-9d655b964b33   in service    fd00:1122:3344:102::5
    oximeter          f4806e03-9ecc-49ef-8ee0-ff7aee4dab23   in service    fd00:1122:3344:102::6


 MODIFIED SLEDS:

  sled a21d85f9-ff13-4800-9e86-5eb7b9e62eb5 (active):

    physical disks at generation 2:
    -------------------------------------------------------------
    vendor             model                serial
    -------------------------------------------------------------
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g0_0
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g0_1
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g0_2
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g0_3
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g0_4


    datasets generation 4 -> 5:
    -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    dataset name                                                                                                   dataset uuid                           quota     reservation   compression
    -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/cockroachdb                                                     119fec8f-785e-4833-90f2-04311332041f   none      none          off
    oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6/crypt/cockroachdb                                                     1f8b0e1a-0545-44f1-9366-6aaf29a0fb83   none      none          off
    oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crucible                                                              aff65822-a39d-4b21-9b1e-d94ba1688057   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crucible                                                              76cc81e2-f740-4cb9-b495-10ae9cbb8cbd   none      none          off
    oxp_44152dd0-a9bc-40a0-a64f-d375b950e5bf/crucible                                                              305cdc80-78d1-4e19-8b68-ba68be9e7b1a   none      none          off
    oxp_abbce0e4-bda7-4853-b89a-37721aa638fb/crucible                                                              a3c3f7cc-b4e4-439b-9eb6-e97ad14b58fe   none      none          off
    oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6/crucible                                                              1c2913e3-adce-4a64-8c46-b47c8018b291   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/external_dns                                                    0712c964-bf3f-45e7-9e75-361652b53736   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/internal_dns                                                    03492708-b9f6-4c40-a0fb-965734468a66   none      none          off
    oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crypt/zone                                                            cac4df64-07ac-4266-9c73-822fb620ff9f   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/zone                                                            da1424d7-fe97-4f3e-84a1-4842ab4dee99   none      none          off
    oxp_44152dd0-a9bc-40a0-a64f-d375b950e5bf/crypt/zone                                                            97c712a4-cb43-4e83-960b-9382964bae74   none      none          off
    oxp_abbce0e4-bda7-4853-b89a-37721aa638fb/crypt/zone                                                            7fe4e11d-6cfb-4e35-973c-fe366a8cd9a1   none      none          off
    oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6/crypt/zone                                                            6f7a280d-2aca-4e64-9ecd-c9432c9a80f4   none      none          off
    oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6/crypt/zone/oxz_cockroachdb_792489e0-9d08-4d21-9caa-0440b12d6ade       2d4851d8-0d00-45a3-b351-9353339e3f17   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/zone/oxz_cockroachdb_83e82c9d-e587-4a68-a18d-128712e2741f       c4351281-bd7d-41bd-b5a1-e9118dac2f16   none      none          off
    oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crypt/zone/oxz_crucible_049d9f96-6e06-43a0-a924-35146efd7b8c          801a8141-9e83-4cc0-9428-fb1db210657d   none      none          off
    oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6/crypt/zone/oxz_crucible_6d03d1db-c605-4a00-9717-3bf867969ad9          1728ac4b-67aa-451f-b674-ad9b60dd5fdb   none      none          off
    oxp_44152dd0-a9bc-40a0-a64f-d375b950e5bf/crypt/zone/oxz_crucible_8ab366b1-b6f7-4467-8468-36745617618c          05146ed5-3246-4591-adec-e8a7941b8e83   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/zone/oxz_crucible_b26d1945-d24b-41f9-b80f-77af3638941a          3d967991-7f45-41f6-9cb5-827b90645547   none      none          off
    oxp_abbce0e4-bda7-4853-b89a-37721aa638fb/crypt/zone/oxz_crucible_b67c6455-b415-4404-b997-588d39ae4a6c          bd097130-13a7-4478-9a0e-e6eadeefabf4   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/zone/oxz_crucible_pantry_8b81d199-146c-4ccf-aa23-f57140a6b582   d4ce5cf6-96e8-44f0-975f-b5601f099c09   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/zone/oxz_external_dns_5c88569a-d13a-483c-b276-90730292d35b      5073be4d-3c52-4948-b7ce-2a708c724d02   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/zone/oxz_internal_dns_c3e14e81-0d5a-4461-80f5-1144ccdf4da5      04128921-a9b4-40f1-bd7a-17e89c22fac9   none      none          off
    oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6/crypt/zone/oxz_nexus_b8da49f4-5b11-4773-be0c-8cbae8f31400             0ab36247-5490-4fe0-b3ab-cbd5d9170efc   none      none          off
    oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crypt/zone/oxz_ntp_2b3c2cf8-bf97-4a7c-9327-712f1d589c7b               43a80037-e23f-44be-84eb-bb30bd1f539e   none      none          off
    oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crypt/debug                                                           6f610524-4329-4634-adab-ffbd6f65a653   100 GiB   none          gzip-9
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/debug                                                           eb609f8e-0bd4-4f07-8c8c-8ad72bc3b8e6   100 GiB   none          gzip-9
    oxp_44152dd0-a9bc-40a0-a64f-d375b950e5bf/crypt/debug                                                           dd9e2a12-9e1a-4c01-9090-2318a4ca5bf6   100 GiB   none          gzip-9
    oxp_abbce0e4-bda7-4853-b89a-37721aa638fb/crypt/debug                                                           87c3fca0-606e-47ff-81cb-0917fe240f85   100 GiB   none          gzip-9
    oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6/crypt/debug                                                           d60fcccf-fc65-404c-a2d9-0c780a7e1ea8   100 GiB   none          gzip-9
*   oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crypt/external_dns                                                    e517594e-729d-40dc-9b05-75f0866cc720   none      none          off
*   oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crypt/zone/oxz_external_dns_13f9e4ae-4f94-4718-af29-4a5da01bac2f      9b229bcf-1e70-4c34-a34f-20c022968645   none      none          off


    omicron zones generation 7 -> 8:
    ----------------------------------------------------------------------------------------------
    zone type         zone id                                disposition    underlay IP
    ----------------------------------------------------------------------------------------------
    boundary_ntp      2b3c2cf8-bf97-4a7c-9327-712f1d589c7b   in service     fd00:1122:3344:101::d
    cockroach_db      792489e0-9d08-4d21-9caa-0440b12d6ade   in service     fd00:1122:3344:101::4
    cockroach_db      83e82c9d-e587-4a68-a18d-128712e2741f   in service     fd00:1122:3344:101::3
    crucible          049d9f96-6e06-43a0-a924-35146efd7b8c   in service     fd00:1122:3344:101::a
    crucible          6d03d1db-c605-4a00-9717-3bf867969ad9   in service     fd00:1122:3344:101::9
    crucible          8ab366b1-b6f7-4467-8468-36745617618c   in service     fd00:1122:3344:101::c
    crucible          b26d1945-d24b-41f9-b80f-77af3638941a   in service     fd00:1122:3344:101::8
    crucible          b67c6455-b415-4404-b997-588d39ae4a6c   in service     fd00:1122:3344:101::b
    crucible_pantry   8b81d199-146c-4ccf-aa23-f57140a6b582   in service     fd00:1122:3344:101::7
    external_dns      5c88569a-d13a-483c-b276-90730292d35b   expunged       fd00:1122:3344:101::5
    internal_dns      c3e14e81-0d5a-4461-80f5-1144ccdf4da5   in service     fd00:1122:3344:1::1
    nexus             b8da49f4-5b11-4773-be0c-8cbae8f31400   in service     fd00:1122:3344:101::6
*   external_dns      13f9e4ae-4f94-4718-af29-4a5da01bac2f   - in service   fd00:1122:3344:101::21
     └─                                                      + expunged


 COCKROACHDB SETTINGS:
    state fingerprint:::::::::::::::::   d4d87aa2ad877a4cc2fddd0573952362739110de (unchanged)
    cluster.preserve_downgrade_option:   "22.1" (unchanged)

 METADATA:
    internal DNS version:   2 (unchanged)
    external DNS version:   2 (unchanged)


internal DNS:
* DNS zone: "control-plane.oxide.internal":
    name: 0208b07b-4ccd-4e8b-b334-1ec8ecc4bae7.host          (records: 1)
        AAAA fd00:1122:3344:103::6
    name: 049d9f96-6e06-43a0-a924-35146efd7b8c.host          (records: 1)
        AAAA fd00:1122:3344:101::a
    name: 0a1d8035-c373-4de7-8d70-ebe0a4b16758.host          (records: 1)
        AAAA fd00:1122:3344:102::c
    name: 0c752586-eb3b-4857-aa35-85a1206b6efb.host          (records: 1)
        AAAA fd00:1122:3344:3::1
-   name: 13f9e4ae-4f94-4718-af29-4a5da01bac2f.host          (records: 1)
-       AAAA fd00:1122:3344:101::21
    name: 23d6123b-1b33-4ea9-8eb0-62607cc29679.host          (records: 1)
        AAAA fd00:1122:3344:2::1
    name: 2b3c2cf8-bf97-4a7c-9327-712f1d589c7b.host          (records: 1)
        AAAA fd00:1122:3344:101::d
    name: 2c00cc83-0211-4662-9f95-ecbf1c8fc26b.host          (records: 1)
        AAAA fd00:1122:3344:102::3
    name: 2eb1e0f2-72a3-48bc-a76e-cb5b8838938f.host          (records: 1)
        AAAA fd00:1122:3344:102::4
    name: 31170c27-6736-418a-b496-b740a5668e4c.host          (records: 1)
        AAAA fd00:1122:3344:103::7
    name: 476d5bde-b360-4c49-8093-5d56be92c63f.host          (records: 1)
        AAAA fd00:1122:3344:102::a
    name: 47d0ca61-9861-4d61-948c-890c5f4ac5d5.host          (records: 1)
        AAAA fd00:1122:3344:102::7
    name: 67989451-63b3-48c2-aef8-c6e511fb122f.host          (records: 1)
        AAAA fd00:1122:3344:103::3
    name: 6d03d1db-c605-4a00-9717-3bf867969ad9.host          (records: 1)
        AAAA fd00:1122:3344:101::9
    name: 74d4206f-6bdb-4c47-b01c-9e2a876d622e.host          (records: 1)
        AAAA fd00:1122:3344:102::9
    name: 792489e0-9d08-4d21-9caa-0440b12d6ade.host          (records: 1)
        AAAA fd00:1122:3344:101::4
    name: 79da9a6d-9db8-4fd9-b2dc-ba0d2fed0bdd.host          (records: 1)
        AAAA fd00:1122:3344:103::8
    name: 7b9677c8-0daf-4ab1-bc1d-9d655b964b33.host          (records: 1)
        AAAA fd00:1122:3344:102::5
    name: 7c5ae2a6-95a7-4cd9-8631-68477b614d97.host          (records: 1)
        AAAA fd00:1122:3344:103::a
    name: 83e82c9d-e587-4a68-a18d-128712e2741f.host          (records: 1)
        AAAA fd00:1122:3344:101::3
    name: 8ab366b1-b6f7-4467-8468-36745617618c.host          (records: 1)
        AAAA fd00:1122:3344:101::c
    name: 8b81d199-146c-4ccf-aa23-f57140a6b582.host          (records: 1)
        AAAA fd00:1122:3344:101::7
    name: 8ea2a399-24f4-44c1-a31c-9b83750de6c1.host          (records: 1)
        AAAA fd00:1122:3344:103::4
    name: _boundary-ntp._tcp                                 (records: 2)
        SRV  port   123 2b3c2cf8-bf97-4a7c-9327-712f1d589c7b.host.control-plane.oxide.internal
        SRV  port   123 f4bcdbb7-c64b-4473-9346-e5c6c411e76d.host.control-plane.oxide.internal
    name: _clickhouse-native._tcp                            (records: 1)
        SRV  port  9000 0208b07b-4ccd-4e8b-b334-1ec8ecc4bae7.host.control-plane.oxide.internal
    name: _clickhouse._tcp                                   (records: 1)
        SRV  port  8123 0208b07b-4ccd-4e8b-b334-1ec8ecc4bae7.host.control-plane.oxide.internal
    name: _cockroach._tcp                                    (records: 5)
        SRV  port 32221 2c00cc83-0211-4662-9f95-ecbf1c8fc26b.host.control-plane.oxide.internal
        SRV  port 32221 2eb1e0f2-72a3-48bc-a76e-cb5b8838938f.host.control-plane.oxide.internal
        SRV  port 32221 67989451-63b3-48c2-aef8-c6e511fb122f.host.control-plane.oxide.internal
        SRV  port 32221 792489e0-9d08-4d21-9caa-0440b12d6ade.host.control-plane.oxide.internal
        SRV  port 32221 83e82c9d-e587-4a68-a18d-128712e2741f.host.control-plane.oxide.internal
    name: _crucible-pantry._tcp                              (records: 3)
        SRV  port 17000 31170c27-6736-418a-b496-b740a5668e4c.host.control-plane.oxide.internal
        SRV  port 17000 47d0ca61-9861-4d61-948c-890c5f4ac5d5.host.control-plane.oxide.internal
        SRV  port 17000 8b81d199-146c-4ccf-aa23-f57140a6b582.host.control-plane.oxide.internal
    name: _crucible._tcp.049d9f96-6e06-43a0-a924-35146efd7b8c (records: 1)
        SRV  port 32345 049d9f96-6e06-43a0-a924-35146efd7b8c.host.control-plane.oxide.internal
    name: _crucible._tcp.0a1d8035-c373-4de7-8d70-ebe0a4b16758 (records: 1)
        SRV  port 32345 0a1d8035-c373-4de7-8d70-ebe0a4b16758.host.control-plane.oxide.internal
    name: _crucible._tcp.476d5bde-b360-4c49-8093-5d56be92c63f (records: 1)
        SRV  port 32345 476d5bde-b360-4c49-8093-5d56be92c63f.host.control-plane.oxide.internal
    name: _crucible._tcp.6d03d1db-c605-4a00-9717-3bf867969ad9 (records: 1)
        SRV  port 32345 6d03d1db-c605-4a00-9717-3bf867969ad9.host.control-plane.oxide.internal
    name: _crucible._tcp.74d4206f-6bdb-4c47-b01c-9e2a876d622e (records: 1)
        SRV  port 32345 74d4206f-6bdb-4c47-b01c-9e2a876d622e.host.control-plane.oxide.internal
    name: _crucible._tcp.79da9a6d-9db8-4fd9-b2dc-ba0d2fed0bdd (records: 1)
        SRV  port 32345 79da9a6d-9db8-4fd9-b2dc-ba0d2fed0bdd.host.control-plane.oxide.internal
    name: _crucible._tcp.7c5ae2a6-95a7-4cd9-8631-68477b614d97 (records: 1)
        SRV  port 32345 7c5ae2a6-95a7-4cd9-8631-68477b614d97.host.control-plane.oxide.internal
    name: _crucible._tcp.8ab366b1-b6f7-4467-8468-36745617618c (records: 1)
        SRV  port 32345 8ab366b1-b6f7-4467-8468-36745617618c.host.control-plane.oxide.internal
    name: _crucible._tcp.a0e1f460-0a7c-4788-b4a2-3ecc2ac909ad (records: 1)
        SRV  port 32345 a0e1f460-0a7c-4788-b4a2-3ecc2ac909ad.host.control-plane.oxide.internal
    name: _crucible._tcp.a3cb63c4-0044-4705-a96f-87d83b09ea5d (records: 1)
        SRV  port 32345 a3cb63c4-0044-4705-a96f-87d83b09ea5d.host.control-plane.oxide.internal
    name: _crucible._tcp.b16765f7-de2f-44c2-b6ae-4fb60f5c4d53 (records: 1)
        SRV  port 32345 b16765f7-de2f-44c2-b6ae-4fb60f5c4d53.host.control-plane.oxide.internal
    name: _crucible._tcp.b26d1945-d24b-41f9-b80f-77af3638941a (records: 1)
        SRV  port 32345 b26d1945-d24b-41f9-b80f-77af3638941a.host.control-plane.oxide.internal
    name: _crucible._tcp.b67c6455-b415-4404-b997-588d39ae4a6c (records: 1)
        SRV  port 32345 b67c6455-b415-4404-b997-588d39ae4a6c.host.control-plane.oxide.internal
    name: _crucible._tcp.c673bafa-fcd2-40c4-a7f9-20906391ba42 (records: 1)
        SRV  port 32345 c673bafa-fcd2-40c4-a7f9-20906391ba42.host.control-plane.oxide.internal
    name: _crucible._tcp.f1497d76-dc9d-40ac-9670-3b17e1a64b84 (records: 1)
        SRV  port 32345 f1497d76-dc9d-40ac-9670-3b17e1a64b84.host.control-plane.oxide.internal
    name: _dendrite._tcp                                     (records: 2)
        SRV  port 12224 dendrite-09301471-54be-465e-a674-45da58c0c59b.host.control-plane.oxide.internal
        SRV  port 12224 dendrite-a21d85f9-ff13-4800-9e86-5eb7b9e62eb5.host.control-plane.oxide.internal
*   name: _external-dns._tcp                                 (records: 2 -> 1)
-       SRV  port  5353 13f9e4ae-4f94-4718-af29-4a5da01bac2f.host.control-plane.oxide.internal
-       SRV  port  5353 8ea2a399-24f4-44c1-a31c-9b83750de6c1.host.control-plane.oxide.internal
+       SRV  port  5353 8ea2a399-24f4-44c1-a31c-9b83750de6c1.host.control-plane.oxide.internal
    name: _internal-ntp._tcp                                 (records: 1)
        SRV  port   123 d2d54e36-5a56-491e-8910-8a2cbe10cce9.host.control-plane.oxide.internal
    name: _mgd._tcp                                          (records: 2)
        SRV  port  4676 dendrite-09301471-54be-465e-a674-45da58c0c59b.host.control-plane.oxide.internal
        SRV  port  4676 dendrite-a21d85f9-ff13-4800-9e86-5eb7b9e62eb5.host.control-plane.oxide.internal
    name: _mgs._tcp                                          (records: 2)
        SRV  port 12225 dendrite-09301471-54be-465e-a674-45da58c0c59b.host.control-plane.oxide.internal
        SRV  port 12225 dendrite-a21d85f9-ff13-4800-9e86-5eb7b9e62eb5.host.control-plane.oxide.internal
    name: _nameservice._tcp                                  (records: 3)
        SRV  port  5353 0c752586-eb3b-4857-aa35-85a1206b6efb.host.control-plane.oxide.internal
        SRV  port  5353 23d6123b-1b33-4ea9-8eb0-62607cc29679.host.control-plane.oxide.internal
        SRV  port  5353 c3e14e81-0d5a-4461-80f5-1144ccdf4da5.host.control-plane.oxide.internal
    name: _nexus._tcp                                        (records: 3)
        SRV  port 12221 7b9677c8-0daf-4ab1-bc1d-9d655b964b33.host.control-plane.oxide.internal
        SRV  port 12221 b8da49f4-5b11-4773-be0c-8cbae8f31400.host.control-plane.oxide.internal
        SRV  port 12221 bdf0578f-d601-44e9-9318-5b5083d3e57d.host.control-plane.oxide.internal
    name: _oximeter._tcp                                     (records: 1)
        SRV  port 12223 f4806e03-9ecc-49ef-8ee0-ff7aee4dab23.host.control-plane.oxide.internal
    name: a0e1f460-0a7c-4788-b4a2-3ecc2ac909ad.host          (records: 1)
        AAAA fd00:1122:3344:103::c
    name: a3cb63c4-0044-4705-a96f-87d83b09ea5d.host          (records: 1)
        AAAA fd00:1122:3344:102::b
    name: b16765f7-de2f-44c2-b6ae-4fb60f5c4d53.host          (records: 1)
        AAAA fd00:1122:3344:103::b
    name: b26d1945-d24b-41f9-b80f-77af3638941a.host          (records: 1)
        AAAA fd00:1122:3344:101::8
    name: b67c6455-b415-4404-b997-588d39ae4a6c.host          (records: 1)
        AAAA fd00:1122:3344:101::b
    name: b8da49f4-5b11-4773-be0c-8cbae8f31400.host          (records: 1)
        AAAA fd00:1122:3344:101::6
    name: bdf0578f-d601-44e9-9318-5b5083d3e57d.host          (records: 1)
        AAAA fd00:1122:3344:103::5
    name: boundary-ntp                                       (records: 2)
        AAAA fd00:1122:3344:101::d
        AAAA fd00:1122:3344:102::d
    name: c3e14e81-0d5a-4461-80f5-1144ccdf4da5.host          (records: 1)
        AAAA fd00:1122:3344:1::1
    name: c673bafa-fcd2-40c4-a7f9-20906391ba42.host          (records: 1)
        AAAA fd00:1122:3344:102::8
    name: d2d54e36-5a56-491e-8910-8a2cbe10cce9.host          (records: 1)
        AAAA fd00:1122:3344:103::d
    name: dendrite-09301471-54be-465e-a674-45da58c0c59b.host (records: 1)
        AAAA fd00:1122:3344:103::2
    name: dendrite-a21d85f9-ff13-4800-9e86-5eb7b9e62eb5.host (records: 1)
        AAAA fd00:1122:3344:101::2
    name: f1497d76-dc9d-40ac-9670-3b17e1a64b84.host          (records: 1)
        AAAA fd00:1122:3344:103::9
    name: f4806e03-9ecc-49ef-8ee0-ff7aee4dab23.host          (records: 1)
        AAAA fd00:1122:3344:102::6
    name: f4bcdbb7-c64b-4473-9346-e5c6c411e76d.host          (records: 1)
        AAAA fd00:1122:3344:102::d

external DNS:
  DNS zone: "oxide.test" (unchanged)
    name: recovery.sys                                       (records: 3)
        A    198.51.100.23
        A    198.51.100.24
        A    198.51.100.22


〉

Execution took almost two minutes for some reason but did complete successfully:

root@oxz_switch:~# omdb -w nexus blueprints import blueprint-external-dns-2-expunged.json
note: Nexus URL not specified.  Will pick one from DNS. 
note: using DNS server for subnet fd00:1122:3344::/48
note: (if this is not right, use --dns-server to specify an alternate DNS server)
note: using Nexus URL http://[fd00:1122:3344:101::6]:12221
uploaded new blueprint 296265f5-ed95-48e4-9eb2-8860c7c006c7

root@oxz_switch:~# omdb -w nexus blueprints target set 296265f5-ed95-48e4-9eb2-8860c7c006c7 inherit
note: Nexus URL not specified.  Will pick one from DNS. 
note: using DNS server for subnet fd00:1122:3344::/48
note: (if this is not right, use --dns-server to specify an alternate DNS server)
note: using Nexus URL http://[fd00:1122:3344:101::6]:12221
set target blueprint to 296265f5-ed95-48e4-9eb2-8860c7c006c7

...

root@oxz_switch:~# omdb nexus background-tasks show blueprint_executor
note: Nexus URL not specified.  Will pick one from DNS. 
note: using DNS server for subnet fd00:1122:3344::/48
note: (if this is not right, use --dns-server to specify an alternate DNS server)
note: using Nexus URL http://[fd00:1122:3344:101::6]:12221
task: "blueprint_executor"
  configured period: every 1m
  currently executing: iter 25727, triggered by a periodic timer firing
    started at 2025-01-06T21:51:42.668Z, running for 7368ms
  last completed activation: iter 25726, triggered by a dependent task completing
    started at 2025-01-06T21:49:45.565Z (124s ago) and ran for 117109ms
    target blueprint: 296265f5-ed95-48e4-9eb2-8860c7c006c7
    execution:        enabled
    status:           completed (16 steps)
    error:            (none)

At this point I expected that if I generated a new blueprint through the planner, it'd add an external DNS zone to the disk it shouldn't. That's what happened:

root@oxz_switch:~# omdb -w nexus blueprints regenerate
note: Nexus URL not specified.  Will pick one from DNS.
note: using DNS server for subnet fd00:1122:3344::/48
note: (if this is not right, use --dns-server to specify an alternate DNS server)
note: using Nexus URL http://[fd00:1122:3344:101::6]:12221
generated new blueprint d655fa49-7ad4-45ca-a5df-db6c1b490cd8


root@oxz_switch:~# omdb nexus blueprints diff current d655fa49-7ad4-45ca-a5df-db6c1b490cd8
note: Nexus URL not specified.  Will pick one from DNS.
note: using DNS server for subnet fd00:1122:3344::/48
note: (if this is not right, use --dns-server to specify an alternate DNS server)
note: using Nexus URL http://[fd00:1122:3344:101::6]:12221
from: blueprint 296265f5-ed95-48e4-9eb2-8860c7c006c7
to:   blueprint d655fa49-7ad4-45ca-a5df-db6c1b490cd8

 UNCHANGED SLEDS:

  sled 09301471-54be-465e-a674-45da58c0c59b (active):

    physical disks at generation 2:
    -------------------------------------------------------------
    vendor             model                serial
    -------------------------------------------------------------
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g3_0
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g3_1
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g3_2
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g3_3
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g3_4


    datasets at generation 2:
    -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    dataset name                                                                                                   dataset uuid                           quota     reservation   compression
    -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/cockroachdb                                                     355ddd2b-b336-45c6-903f-ecdc99b26289   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crucible                                                              36b554b7-d442-4c58-96fe-71f88d9145e6   none      none          off
    oxp_29f71bd8-8d07-44f9-a587-8dd6552d6b3d/crucible                                                              53b65adf-99c3-472b-849c-9385e61c7f80   none      none          off
    oxp_639dcdf6-d036-4e59-94d8-9a05706f2a6e/crucible                                                              ee41d385-8bc3-4932-971a-2f2b1b3461a1   none      none          off
    oxp_b5c23cec-84ac-4810-9077-3f960af72fcd/crucible                                                              ab04e945-3a16-4a33-8e8b-a87b7d51df6f   none      none          off
    oxp_b9dc2f06-bf04-463c-8ffe-79d8357d852b/crucible                                                              9f46899f-ee8c-46e1-b6a3-99f74974379f   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/clickhouse                                                      ff1f90df-76b9-46a1-b2aa-c960db481817   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/external_dns                                                    5c81c9ca-43b3-43a2-ae34-2b2d09544492   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/internal_dns                                                    c2df2f3e-9b96-48ec-bcf2-23ffccbc3c05   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/zone                                                            0e6b946e-f23d-402d-87d0-6531ef26c24a   none      none          off
    oxp_29f71bd8-8d07-44f9-a587-8dd6552d6b3d/crypt/zone                                                            48566631-3eba-4cd5-b4e9-48bb9657d8de   none      none          off
    oxp_639dcdf6-d036-4e59-94d8-9a05706f2a6e/crypt/zone                                                            644e63c6-ff9b-4ac0-a932-b1063b2a0a02   none      none          off
    oxp_b5c23cec-84ac-4810-9077-3f960af72fcd/crypt/zone                                                            c04b17c8-5626-409a-8eb1-f0b797fe9788   none      none          off
    oxp_b9dc2f06-bf04-463c-8ffe-79d8357d852b/crypt/zone                                                            f760e7e6-089d-4743-a339-1e0d882a40aa   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/zone/oxz_clickhouse_0208b07b-4ccd-4e8b-b334-1ec8ecc4bae7        d343db99-4aa1-4f6c-aa53-7be34e509d36   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/zone/oxz_cockroachdb_67989451-63b3-48c2-aef8-c6e511fb122f       9c747a15-0ce0-41bd-bc2f-d7ee792a3592   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/zone/oxz_crucible_79da9a6d-9db8-4fd9-b2dc-ba0d2fed0bdd          6ec06bb2-9fd0-45d0-a35b-d6cbf894537d   none      none          off
    oxp_639dcdf6-d036-4e59-94d8-9a05706f2a6e/crypt/zone/oxz_crucible_7c5ae2a6-95a7-4cd9-8631-68477b614d97          a4dd3c84-8616-4182-8214-c9677a57318e   none      none          off
    oxp_29f71bd8-8d07-44f9-a587-8dd6552d6b3d/crypt/zone/oxz_crucible_a0e1f460-0a7c-4788-b4a2-3ecc2ac909ad          a93a6377-6d78-4c34-b220-f2c4926e4795   none      none          off
    oxp_b5c23cec-84ac-4810-9077-3f960af72fcd/crypt/zone/oxz_crucible_b16765f7-de2f-44c2-b6ae-4fb60f5c4d53          660a8beb-683a-4c00-82e9-b8f19e9f86fe   none      none          off
    oxp_b9dc2f06-bf04-463c-8ffe-79d8357d852b/crypt/zone/oxz_crucible_f1497d76-dc9d-40ac-9670-3b17e1a64b84          37896114-bb11-402a-a1a2-f0fe800d794d   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/zone/oxz_crucible_pantry_31170c27-6736-418a-b496-b740a5668e4c   7fc29a07-ce43-41d9-9a1e-1bfd3be35a46   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/zone/oxz_external_dns_8ea2a399-24f4-44c1-a31c-9b83750de6c1      defada08-7725-4d45-acec-2fe308bae2e7   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/zone/oxz_internal_dns_0c752586-eb3b-4857-aa35-85a1206b6efb      28d664b6-8003-489b-b824-262025837f56   none      none          off
    oxp_29f71bd8-8d07-44f9-a587-8dd6552d6b3d/crypt/zone/oxz_nexus_bdf0578f-d601-44e9-9318-5b5083d3e57d             30179c95-0623-468d-8b25-b78499725884   none      none          off
    oxp_639dcdf6-d036-4e59-94d8-9a05706f2a6e/crypt/zone/oxz_ntp_d2d54e36-5a56-491e-8910-8a2cbe10cce9               c6649371-7506-4fa3-ad41-f4d7666bc1a3   none      none          off
    oxp_13a7b934-412b-4ea5-8391-d52998c2c6fe/crypt/debug                                                           50e80495-eb69-4716-8fa4-9f697b454498   100 GiB   none          gzip-9
    oxp_29f71bd8-8d07-44f9-a587-8dd6552d6b3d/crypt/debug                                                           f8a339ec-c736-44cd-b562-d845da329ece   100 GiB   none          gzip-9
    oxp_639dcdf6-d036-4e59-94d8-9a05706f2a6e/crypt/debug                                                           7f64c446-bc62-4885-8d74-7a40906c6f20   100 GiB   none          gzip-9
    oxp_b5c23cec-84ac-4810-9077-3f960af72fcd/crypt/debug                                                           59d97482-de4f-4f5a-9961-d2b00126a6a6   100 GiB   none          gzip-9
    oxp_b9dc2f06-bf04-463c-8ffe-79d8357d852b/crypt/debug                                                           914cc29f-b2cd-4128-9393-18e8d6715730   100 GiB   none          gzip-9


    omicron zones at generation 5:
    --------------------------------------------------------------------------------------------
    zone type         zone id                                disposition   underlay IP
    --------------------------------------------------------------------------------------------
    clickhouse        0208b07b-4ccd-4e8b-b334-1ec8ecc4bae7   in service    fd00:1122:3344:103::6
    cockroach_db      67989451-63b3-48c2-aef8-c6e511fb122f   in service    fd00:1122:3344:103::3
    crucible          79da9a6d-9db8-4fd9-b2dc-ba0d2fed0bdd   in service    fd00:1122:3344:103::8
    crucible          7c5ae2a6-95a7-4cd9-8631-68477b614d97   in service    fd00:1122:3344:103::a
    crucible          a0e1f460-0a7c-4788-b4a2-3ecc2ac909ad   in service    fd00:1122:3344:103::c
    crucible          b16765f7-de2f-44c2-b6ae-4fb60f5c4d53   in service    fd00:1122:3344:103::b
    crucible          f1497d76-dc9d-40ac-9670-3b17e1a64b84   in service    fd00:1122:3344:103::9
    crucible_pantry   31170c27-6736-418a-b496-b740a5668e4c   in service    fd00:1122:3344:103::7
    external_dns      8ea2a399-24f4-44c1-a31c-9b83750de6c1   in service    fd00:1122:3344:103::4
    internal_dns      0c752586-eb3b-4857-aa35-85a1206b6efb   in service    fd00:1122:3344:3::1
    internal_ntp      d2d54e36-5a56-491e-8910-8a2cbe10cce9   in service    fd00:1122:3344:103::d
    nexus             bdf0578f-d601-44e9-9318-5b5083d3e57d   in service    fd00:1122:3344:103::5


  sled f0b13ea8-c304-4a03-ab7c-da1fb2d0c612 (active):

    physical disks at generation 2:
    -------------------------------------------------------------
    vendor             model                serial
    -------------------------------------------------------------
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g1_0
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g1_1
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g1_2
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g1_3
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g1_4


    datasets at generation 2:
    -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    dataset name                                                                                                   dataset uuid                           quota     reservation   compression
    -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/cockroachdb                                                     2cfac653-9dba-492a-b4e5-c8efe5a1e1b9   none      none          off
    oxp_9d453deb-a057-438f-b555-627c501d9257/crypt/cockroachdb                                                     12feda67-cf20-4f01-8094-69a956526990   none      none          off
    oxp_60155e33-c180-4490-9f28-0cb4003ede47/crucible                                                              d597754a-6fe3-4d18-b8be-8dc0a269628d   none      none          off
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crucible                                                              beee4d0a-44b9-4298-9014-da523dd9eaa7   none      none          off
    oxp_9d453deb-a057-438f-b555-627c501d9257/crucible                                                              644161d1-8a5f-4609-ae2d-21c4ae54fdd4   none      none          off
    oxp_c1b87796-8265-4c6b-b41a-be7de82934f6/crucible                                                              5d8edd26-0cd7-434e-af7b-c50a319cd1ea   none      none          off
    oxp_f63bc936-be56-4092-a2cb-6a8508f8a23d/crucible                                                              cdf87aea-8f8f-4e49-b921-db848d6e25ef   none      none          off
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/internal_dns                                                    b1e1c105-638d-4401-a5af-534f219a11f4   none      none          off
    oxp_60155e33-c180-4490-9f28-0cb4003ede47/crypt/zone                                                            d621f634-68d8-4741-85bd-53e82095c094   none      none          off
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/zone                                                            7c00d8d6-86c6-4446-a90f-24c62ac741c4   none      none          off
    oxp_9d453deb-a057-438f-b555-627c501d9257/crypt/zone                                                            b5fcfc17-2954-484a-bbf8-8c51b97a7aa3   none      none          off
    oxp_c1b87796-8265-4c6b-b41a-be7de82934f6/crypt/zone                                                            aacb001f-2c30-427d-96cf-10f553de5446   none      none          off
    oxp_f63bc936-be56-4092-a2cb-6a8508f8a23d/crypt/zone                                                            662d12c6-b68b-4aac-b9be-9c999bf5b05e   none      none          off
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/zone/oxz_cockroachdb_2c00cc83-0211-4662-9f95-ecbf1c8fc26b       1478643a-989f-43e3-bc24-4ab2c9f40d15   none      none          off
    oxp_9d453deb-a057-438f-b555-627c501d9257/crypt/zone/oxz_cockroachdb_2eb1e0f2-72a3-48bc-a76e-cb5b8838938f       905d06de-7198-41a7-a696-35d1c66bc882   none      none          off
    oxp_60155e33-c180-4490-9f28-0cb4003ede47/crypt/zone/oxz_crucible_0a1d8035-c373-4de7-8d70-ebe0a4b16758          ac0ca38c-128a-42a5-b87a-670d7b807ed4   none      none          off
    oxp_f63bc936-be56-4092-a2cb-6a8508f8a23d/crypt/zone/oxz_crucible_476d5bde-b360-4c49-8093-5d56be92c63f          d642b0d0-1f61-4ad0-9840-98464d3b4ba2   none      none          off
    oxp_9d453deb-a057-438f-b555-627c501d9257/crypt/zone/oxz_crucible_74d4206f-6bdb-4c47-b01c-9e2a876d622e          19a81e6a-938a-4579-bf95-e1105fe1b0a5   none      none          off
    oxp_c1b87796-8265-4c6b-b41a-be7de82934f6/crypt/zone/oxz_crucible_a3cb63c4-0044-4705-a96f-87d83b09ea5d          7df8ef54-cb4a-4676-b163-7d0418043127   none      none          off
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/zone/oxz_crucible_c673bafa-fcd2-40c4-a7f9-20906391ba42          88a432f9-1f5f-4799-a303-ab5806f73f3f   none      none          off
    oxp_60155e33-c180-4490-9f28-0cb4003ede47/crypt/zone/oxz_crucible_pantry_47d0ca61-9861-4d61-948c-890c5f4ac5d5   3cbb1ed7-3290-428d-bebb-f54297c09909   none      none          off
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/zone/oxz_internal_dns_23d6123b-1b33-4ea9-8eb0-62607cc29679      74b11e9a-bf6a-458d-a2c7-740576494cc9   none      none          off
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/zone/oxz_nexus_7b9677c8-0daf-4ab1-bc1d-9d655b964b33             0d21ae90-6ee4-4422-9bc6-f336fd4b9ff5   none      none          off
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/zone/oxz_ntp_f4bcdbb7-c64b-4473-9346-e5c6c411e76d               7a9e077f-5512-4f8c-b780-36c326aa0ad5   none      none          off
    oxp_60155e33-c180-4490-9f28-0cb4003ede47/crypt/zone/oxz_oximeter_f4806e03-9ecc-49ef-8ee0-ff7aee4dab23          e48d0b5a-2a9e-4794-b45c-858bf8ebf833   none      none          off
    oxp_60155e33-c180-4490-9f28-0cb4003ede47/crypt/debug                                                           5f96f85f-15df-4b76-bc99-7cdd1e07ff16   100 GiB   none          gzip-9
    oxp_89021d00-c808-49ec-ada3-427646a63b46/crypt/debug                                                           5aa39aed-b571-4c7d-b8ae-040c64cfe25d   100 GiB   none          gzip-9
    oxp_9d453deb-a057-438f-b555-627c501d9257/crypt/debug                                                           45fcfb99-bc89-4c87-b5c6-ee0320d3420c   100 GiB   none          gzip-9
    oxp_c1b87796-8265-4c6b-b41a-be7de82934f6/crypt/debug                                                           d8ffad03-2ea6-432f-beb2-a5b2fc4bf885   100 GiB   none          gzip-9
    oxp_f63bc936-be56-4092-a2cb-6a8508f8a23d/crypt/debug                                                           9156026c-38ed-4ee5-b49f-b4682b7a2702   100 GiB   none          gzip-9


    omicron zones at generation 5:
    --------------------------------------------------------------------------------------------
    zone type         zone id                                disposition   underlay IP
    --------------------------------------------------------------------------------------------
    boundary_ntp      f4bcdbb7-c64b-4473-9346-e5c6c411e76d   in service    fd00:1122:3344:102::d
    cockroach_db      2c00cc83-0211-4662-9f95-ecbf1c8fc26b   in service    fd00:1122:3344:102::3
    cockroach_db      2eb1e0f2-72a3-48bc-a76e-cb5b8838938f   in service    fd00:1122:3344:102::4
    crucible          0a1d8035-c373-4de7-8d70-ebe0a4b16758   in service    fd00:1122:3344:102::c
    crucible          476d5bde-b360-4c49-8093-5d56be92c63f   in service    fd00:1122:3344:102::a
    crucible          74d4206f-6bdb-4c47-b01c-9e2a876d622e   in service    fd00:1122:3344:102::9
    crucible          a3cb63c4-0044-4705-a96f-87d83b09ea5d   in service    fd00:1122:3344:102::b
    crucible          c673bafa-fcd2-40c4-a7f9-20906391ba42   in service    fd00:1122:3344:102::8
    crucible_pantry   47d0ca61-9861-4d61-948c-890c5f4ac5d5   in service    fd00:1122:3344:102::7
    internal_dns      23d6123b-1b33-4ea9-8eb0-62607cc29679   in service    fd00:1122:3344:2::1
    nexus             7b9677c8-0daf-4ab1-bc1d-9d655b964b33   in service    fd00:1122:3344:102::5
    oximeter          f4806e03-9ecc-49ef-8ee0-ff7aee4dab23   in service    fd00:1122:3344:102::6


 MODIFIED SLEDS:

  sled a21d85f9-ff13-4800-9e86-5eb7b9e62eb5 (active):

    physical disks at generation 2:
    -------------------------------------------------------------
    vendor             model                serial
    -------------------------------------------------------------
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g0_0
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g0_1
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g0_2
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g0_3
    synthetic-vendor   synthetic-model-U2   synthetic-serial-g0_4


    datasets generation 5 -> 6:
    -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    dataset name                                                                                                   dataset uuid                           quota     reservation   compression
    -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/cockroachdb                                                     119fec8f-785e-4833-90f2-04311332041f   none      none          off
    oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6/crypt/cockroachdb                                                     1f8b0e1a-0545-44f1-9366-6aaf29a0fb83   none      none          off
    oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crucible                                                              aff65822-a39d-4b21-9b1e-d94ba1688057   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crucible                                                              76cc81e2-f740-4cb9-b495-10ae9cbb8cbd   none      none          off
    oxp_44152dd0-a9bc-40a0-a64f-d375b950e5bf/crucible                                                              305cdc80-78d1-4e19-8b68-ba68be9e7b1a   none      none          off
    oxp_abbce0e4-bda7-4853-b89a-37721aa638fb/crucible                                                              a3c3f7cc-b4e4-439b-9eb6-e97ad14b58fe   none      none          off
    oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6/crucible                                                              1c2913e3-adce-4a64-8c46-b47c8018b291   none      none          off
    oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crypt/external_dns                                                    e517594e-729d-40dc-9b05-75f0866cc720   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/external_dns                                                    0712c964-bf3f-45e7-9e75-361652b53736   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/internal_dns                                                    03492708-b9f6-4c40-a0fb-965734468a66   none      none          off
    oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crypt/zone                                                            cac4df64-07ac-4266-9c73-822fb620ff9f   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/zone                                                            da1424d7-fe97-4f3e-84a1-4842ab4dee99   none      none          off
    oxp_44152dd0-a9bc-40a0-a64f-d375b950e5bf/crypt/zone                                                            97c712a4-cb43-4e83-960b-9382964bae74   none      none          off
    oxp_abbce0e4-bda7-4853-b89a-37721aa638fb/crypt/zone                                                            7fe4e11d-6cfb-4e35-973c-fe366a8cd9a1   none      none          off
    oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6/crypt/zone                                                            6f7a280d-2aca-4e64-9ecd-c9432c9a80f4   none      none          off
    oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6/crypt/zone/oxz_cockroachdb_792489e0-9d08-4d21-9caa-0440b12d6ade       2d4851d8-0d00-45a3-b351-9353339e3f17   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/zone/oxz_cockroachdb_83e82c9d-e587-4a68-a18d-128712e2741f       c4351281-bd7d-41bd-b5a1-e9118dac2f16   none      none          off
    oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crypt/zone/oxz_crucible_049d9f96-6e06-43a0-a924-35146efd7b8c          801a8141-9e83-4cc0-9428-fb1db210657d   none      none          off
    oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6/crypt/zone/oxz_crucible_6d03d1db-c605-4a00-9717-3bf867969ad9          1728ac4b-67aa-451f-b674-ad9b60dd5fdb   none      none          off
    oxp_44152dd0-a9bc-40a0-a64f-d375b950e5bf/crypt/zone/oxz_crucible_8ab366b1-b6f7-4467-8468-36745617618c          05146ed5-3246-4591-adec-e8a7941b8e83   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/zone/oxz_crucible_b26d1945-d24b-41f9-b80f-77af3638941a          3d967991-7f45-41f6-9cb5-827b90645547   none      none          off
    oxp_abbce0e4-bda7-4853-b89a-37721aa638fb/crypt/zone/oxz_crucible_b67c6455-b415-4404-b997-588d39ae4a6c          bd097130-13a7-4478-9a0e-e6eadeefabf4   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/zone/oxz_crucible_pantry_8b81d199-146c-4ccf-aa23-f57140a6b582   d4ce5cf6-96e8-44f0-975f-b5601f099c09   none      none          off
    oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crypt/zone/oxz_external_dns_13f9e4ae-4f94-4718-af29-4a5da01bac2f      9b229bcf-1e70-4c34-a34f-20c022968645   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/zone/oxz_external_dns_5c88569a-d13a-483c-b276-90730292d35b      5073be4d-3c52-4948-b7ce-2a708c724d02   none      none          off
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/zone/oxz_internal_dns_c3e14e81-0d5a-4461-80f5-1144ccdf4da5      04128921-a9b4-40f1-bd7a-17e89c22fac9   none      none          off
    oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6/crypt/zone/oxz_nexus_b8da49f4-5b11-4773-be0c-8cbae8f31400             0ab36247-5490-4fe0-b3ab-cbd5d9170efc   none      none          off
    oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crypt/zone/oxz_ntp_2b3c2cf8-bf97-4a7c-9327-712f1d589c7b               43a80037-e23f-44be-84eb-bb30bd1f539e   none      none          off
    oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crypt/debug                                                           6f610524-4329-4634-adab-ffbd6f65a653   100 GiB   none          gzip-9
    oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc/crypt/debug                                                           eb609f8e-0bd4-4f07-8c8c-8ad72bc3b8e6   100 GiB   none          gzip-9
    oxp_44152dd0-a9bc-40a0-a64f-d375b950e5bf/crypt/debug                                                           dd9e2a12-9e1a-4c01-9090-2318a4ca5bf6   100 GiB   none          gzip-9
    oxp_abbce0e4-bda7-4853-b89a-37721aa638fb/crypt/debug                                                           87c3fca0-606e-47ff-81cb-0917fe240f85   100 GiB   none          gzip-9
    oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6/crypt/debug                                                           d60fcccf-fc65-404c-a2d9-0c780a7e1ea8   100 GiB   none          gzip-9
+   oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crypt/external_dns                                                    5907b41a-c6ef-4727-bca2-c76c4cd3d0b0   none      none          off
+   oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crypt/zone/oxz_external_dns_218de6fc-3b1e-4be2-93fa-c01aee982262      248b026e-c3dc-4a52-9855-276f06845b56   none      none          off


    omicron zones generation 8 -> 9:
    ---------------------------------------------------------------------------------------------
    zone type         zone id                                disposition   underlay IP
    ---------------------------------------------------------------------------------------------
    boundary_ntp      2b3c2cf8-bf97-4a7c-9327-712f1d589c7b   in service    fd00:1122:3344:101::d
    cockroach_db      792489e0-9d08-4d21-9caa-0440b12d6ade   in service    fd00:1122:3344:101::4
    cockroach_db      83e82c9d-e587-4a68-a18d-128712e2741f   in service    fd00:1122:3344:101::3
    crucible          049d9f96-6e06-43a0-a924-35146efd7b8c   in service    fd00:1122:3344:101::a
    crucible          6d03d1db-c605-4a00-9717-3bf867969ad9   in service    fd00:1122:3344:101::9
    crucible          8ab366b1-b6f7-4467-8468-36745617618c   in service    fd00:1122:3344:101::c
    crucible          b26d1945-d24b-41f9-b80f-77af3638941a   in service    fd00:1122:3344:101::8
    crucible          b67c6455-b415-4404-b997-588d39ae4a6c   in service    fd00:1122:3344:101::b
    crucible_pantry   8b81d199-146c-4ccf-aa23-f57140a6b582   in service    fd00:1122:3344:101::7
    external_dns      13f9e4ae-4f94-4718-af29-4a5da01bac2f   expunged      fd00:1122:3344:101::21
    external_dns      5c88569a-d13a-483c-b276-90730292d35b   expunged      fd00:1122:3344:101::5
    internal_dns      c3e14e81-0d5a-4461-80f5-1144ccdf4da5   in service    fd00:1122:3344:1::1
    nexus             b8da49f4-5b11-4773-be0c-8cbae8f31400   in service    fd00:1122:3344:101::6
+   external_dns      218de6fc-3b1e-4be2-93fa-c01aee982262   in service    fd00:1122:3344:101::22


 COCKROACHDB SETTINGS:
    state fingerprint:::::::::::::::::   d4d87aa2ad877a4cc2fddd0573952362739110de (unchanged)
    cluster.preserve_downgrade_option:   "22.1" (unchanged)

 METADATA:
*   internal DNS version:   2 -> 4
    external DNS version:   2 (unchanged)

Notice that in the diff output above, under datasets, oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crypt/external_dns (the durable dataset for an external DNS zone) appears twice: once for the expunged zone, and once for the zone we're trying to add. They have different ids.

Thankfully, this did fail at execution time:

root@oxz_switch:~# omdb -w nexus blueprints target set d655fa49-7ad4-45ca-a5df-db6c1b490cd8 inherit
note: Nexus URL not specified.  Will pick one from DNS.
note: using DNS server for subnet fd00:1122:3344::/48
note: (if this is not right, use --dns-server to specify an alternate DNS server)
note: using Nexus URL http://[fd00:1122:3344:101::6]:12221
set target blueprint to d655fa49-7ad4-45ca-a5df-db6c1b490cd8


root@oxz_switch:~# omdb nexus background-tasks show blueprint_executor
note: Nexus URL not specified.  Will pick one from DNS.
note: using DNS server for subnet fd00:1122:3344::/48
note: (if this is not right, use --dns-server to specify an alternate DNS server)
note: using Nexus URL http://[fd00:1122:3344:101::6]:12221
task: "blueprint_executor"
  configured period: every 1m
  currently executing: no
  last completed activation: iter 25732, triggered by a dependent task completing
    started at 2025-01-06T21:56:29.734Z (7s ago) and ran for 1776ms
    target blueprint: d655fa49-7ad4-45ca-a5df-db6c1b490cd8
    execution:        enabled
    status:           failed at: Deploy datasets (step 4/16)
    error:            step failed: Deploy datasets
      caused by:      failure deploying datasets: [DatasetManagementStatus { dataset_name: DatasetName { kind: ExternalDns, pool_name: ZpoolName { id: 15b53b30-72cf-4edb-a7c4-325ee3f7c679 (zpool), kind: External } }, err: Some("Dataset oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679/crypt/external_dns exists with a different uuid (has e517594e-729d-40dc-9b05-75f0866cc720, requested 5907b41a-c6ef-4727-bca2-c76c4cd3d0b0)") }]

This makes sense: it found the dataset id associated with the durable dataset of the expunged zone (since we never deleted its dataset), but it was expecting to find the one from the newly-added zone. (This check prevented us from just re-using the old dataset for the new zone!)

Unfortunately, it did update the ledger to reflect the new dataset config generation 6, even though it's invalid:

# cat /pool/int/28d71fca-496b-4b8c-ac69-6b14a14174e5/config/omicron-datasets.json
{"generation":6,"datasets":{"03492708-b9f6-4c40-a0fb-965734468a66":{"id":"03492708-b9f6-4c40-a0fb-965734468a66","name":{"pool_name":"oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc","kind":"internal_dns"},"compression":{"type":"off"},"quota":null,"reservation":null},"04128921-a9b4-40f1-bd7a-17e89c22fac9":{"id":"04128921-a9b4-40f1-bd7a-17e89c22fac9","name":{"pool_name":"oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc","kind":"zone/oxz_internal_dns_c3e14e81-0d5a-4461-80f5-1144ccdf4da5"},"compression":{"type":"off"},"quota":null,"reservation":null},"05146ed5-3246-4591-adec-e8a7941b8e83":{"id":"05146ed5-3246-4591-adec-e8a7941b8e83","name":{"pool_name":"oxp_44152dd0-a9bc-40a0-a64f-d375b950e5bf","kind":"zone/oxz_crucible_8ab366b1-b6f7-4467-8468-36745617618c"},"compression":{"type":"off"},"quota":null,"reservation":null},"0712c964-bf3f-45e7-9e75-361652b53736":{"id":"0712c964-bf3f-45e7-9e75-361652b53736","name":{"pool_name":"oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc","kind":"external_dns"},"compression":{"type":"off"},"quota":null,"reservation":null},"0ab36247-5490-4fe0-b3ab-cbd5d9170efc":{"id":"0ab36247-5490-4fe0-b3ab-cbd5d9170efc","name":{"pool_name":"oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6","kind":"zone/oxz_nexus_b8da49f4-5b11-4773-be0c-8cbae8f31400"},"compression":{"type":"off"},"quota":null,"reservation":null},"119fec8f-785e-4833-90f2-04311332041f":{"id":"119fec8f-785e-4833-90f2-04311332041f","name":{"pool_name":"oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc","kind":"cockroachdb"},"compression":{"type":"off"},"quota":null,"reservation":null},"1728ac4b-67aa-451f-b674-ad9b60dd5fdb":{"id":"1728ac4b-67aa-451f-b674-ad9b60dd5fdb","name":{"pool_name":"oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6","kind":"zone/oxz_crucible_6d03d1db-c605-4a00-9717-3bf867969ad9"},"compression":{"type":"off"},"quota":null,"reservation":null},"1c2913e3-adce-4a64-8c46-b47c8018b291":{"id":"1c2913e3-adce-4a64-8c46-b47c8018b291","name":{"pool_name":"oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6","kind":"crucible"},"compression":{"type":"off"},"quota":null,"reservation":null},"1f8b0e1a-0545-44f1-9366-6aaf29a0fb83":{"id":"1f8b0e1a-0545-44f1-9366-6aaf29a0fb83","name":{"pool_name":"oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6","kind":"cockroachdb"},"compression":{"type":"off"},"quota":null,"reservation":null},"248b026e-c3dc-4a52-9855-276f06845b56":{"id":"248b026e-c3dc-4a52-9855-276f06845b56","name":{"pool_name":"oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679","kind":"zone/oxz_external_dns_218de6fc-3b1e-4be2-93fa-c01aee982262"},"compression":{"type":"off"},"quota":null,"reservation":null},"2d4851d8-0d00-45a3-b351-9353339e3f17":{"id":"2d4851d8-0d00-45a3-b351-9353339e3f17","name":{"pool_name":"oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6","kind":"zone/oxz_cockroachdb_792489e0-9d08-4d21-9caa-0440b12d6ade"},"compression":{"type":"off"},"quota":null,"reservation":null},"305cdc80-78d1-4e19-8b68-ba68be9e7b1a":{"id":"305cdc80-78d1-4e19-8b68-ba68be9e7b1a","name":{"pool_name":"oxp_44152dd0-a9bc-40a0-a64f-d375b950e5bf","kind":"crucible"},"compression":{"type":"off"},"quota":null,"reservation":null},"3d967991-7f45-41f6-9cb5-827b90645547":{"id":"3d967991-7f45-41f6-9cb5-827b90645547","name":{"pool_name":"oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc","kind":"zone/oxz_crucible_b26d1945-d24b-41f9-b80f-77af3638941a"},"compression":{"type":"off"},"quota":null,"reservation":null},"43a80037-e23f-44be-84eb-bb30bd1f539e":{"id":"43a80037-e23f-44be-84eb-bb30bd1f539e","name":{"pool_name":"oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679","kind":"zone/oxz_ntp_2b3c2cf8-bf97-4a7c-9327-712f1d589c7b"},"compression":{"type":"off"},"quota":null,"reservation":null},"5073be4d-3c52-4948-b7ce-2a708c724d02":{"id":"5073be4d-3c52-4948-b7ce-2a708c724d02","name":{"pool_name":"oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc","kind":"zone/oxz_external_dns_5c88569a-d13a-483c-b276-90730292d35b"},"compression":{"type":"off"},"quota":null,"reservation":null},"5907b41a-c6ef-4727-bca2-c76c4cd3d0b0":{"id":"5907b41a-c6ef-4727-bca2-c76c4cd3d0b0","name":{"pool_name":"oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679","kind":"external_dns"},"compression":{"type":"off"},"quota":null,"reservation":null},"6f610524-4329-4634-adab-ffbd6f65a653":{"id":"6f610524-4329-4634-adab-ffbd6f65a653","name":{"pool_name":"oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679","kind":"debug"},"compression":{"type":"gzip_n","level":9},"quota":107374182400,"reservation":null},"6f7a280d-2aca-4e64-9ecd-c9432c9a80f4":{"id":"6f7a280d-2aca-4e64-9ecd-c9432c9a80f4","name":{"pool_name":"oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6","kind":"zone"},"compression":{"type":"off"},"quota":null,"reservation":null},"76cc81e2-f740-4cb9-b495-10ae9cbb8cbd":{"id":"76cc81e2-f740-4cb9-b495-10ae9cbb8cbd","name":{"pool_name":"oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc","kind":"crucible"},"compression":{"type":"off"},"quota":null,"reservation":null},"7fe4e11d-6cfb-4e35-973c-fe366a8cd9a1":{"id":"7fe4e11d-6cfb-4e35-973c-fe366a8cd9a1","name":{"pool_name":"oxp_abbce0e4-bda7-4853-b89a-37721aa638fb","kind":"zone"},"compression":{"type":"off"},"quota":null,"reservation":null},"801a8141-9e83-4cc0-9428-fb1db210657d":{"id":"801a8141-9e83-4cc0-9428-fb1db210657d","name":{"pool_name":"oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679","kind":"zone/oxz_crucible_049d9f96-6e06-43a0-a924-35146efd7b8c"},"compression":{"type":"off"},"quota":null,"reservation":null},"87c3fca0-606e-47ff-81cb-0917fe240f85":{"id":"87c3fca0-606e-47ff-81cb-0917fe240f85","name":{"pool_name":"oxp_abbce0e4-bda7-4853-b89a-37721aa638fb","kind":"debug"},"compression":{"type":"gzip_n","level":9},"quota":107374182400,"reservation":null},"97c712a4-cb43-4e83-960b-9382964bae74":{"id":"97c712a4-cb43-4e83-960b-9382964bae74","name":{"pool_name":"oxp_44152dd0-a9bc-40a0-a64f-d375b950e5bf","kind":"zone"},"compression":{"type":"off"},"quota":null,"reservation":null},"9b229bcf-1e70-4c34-a34f-20c022968645":{"id":"9b229bcf-1e70-4c34-a34f-20c022968645","name":{"pool_name":"oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679","kind":"zone/oxz_external_dns_13f9e4ae-4f94-4718-af29-4a5da01bac2f"},"compression":{"type":"off"},"quota":null,"reservation":null},"a3c3f7cc-b4e4-439b-9eb6-e97ad14b58fe":{"id":"a3c3f7cc-b4e4-439b-9eb6-e97ad14b58fe","name":{"pool_name":"oxp_abbce0e4-bda7-4853-b89a-37721aa638fb","kind":"crucible"},"compression":{"type":"off"},"quota":null,"reservation":null},"aff65822-a39d-4b21-9b1e-d94ba1688057":{"id":"aff65822-a39d-4b21-9b1e-d94ba1688057","name":{"pool_name":"oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679","kind":"crucible"},"compression":{"type":"off"},"quota":null,"reservation":null},"bd097130-13a7-4478-9a0e-e6eadeefabf4":{"id":"bd097130-13a7-4478-9a0e-e6eadeefabf4","name":{"pool_name":"oxp_abbce0e4-bda7-4853-b89a-37721aa638fb","kind":"zone/oxz_crucible_b67c6455-b415-4404-b997-588d39ae4a6c"},"compression":{"type":"off"},"quota":null,"reservation":null},"c4351281-bd7d-41bd-b5a1-e9118dac2f16":{"id":"c4351281-bd7d-41bd-b5a1-e9118dac2f16","name":{"pool_name":"oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc","kind":"zone/oxz_cockroachdb_83e82c9d-e587-4a68-a18d-128712e2741f"},"compression":{"type":"off"},"quota":null,"reservation":null},"cac4df64-07ac-4266-9c73-822fb620ff9f":{"id":"cac4df64-07ac-4266-9c73-822fb620ff9f","name":{"pool_name":"oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679","kind":"zone"},"compression":{"type":"off"},"quota":null,"reservation":null},"d4ce5cf6-96e8-44f0-975f-b5601f099c09":{"id":"d4ce5cf6-96e8-44f0-975f-b5601f099c09","name":{"pool_name":"oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc","kind":"zone/oxz_crucible_pantry_8b81d199-146c-4ccf-aa23-f57140a6b582"},"compression":{"type":"off"},"quota":null,"reservation":null},"d60fcccf-fc65-404c-a2d9-0c780a7e1ea8":{"id":"d60fcccf-fc65-404c-a2d9-0c780a7e1ea8","name":{"pool_name":"oxp_fb97a84c-3ac3-494c-9c76-26d6357aebf6","kind":"debug"},"compression":{"type":"gzip_n","level":9},"quota":107374182400,"reservation":null},"da1424d7-fe97-4f3e-84a1-4842ab4dee99":{"id":"da1424d7-fe97-4f3e-84a1-4842ab4dee99","name":{"pool_name":"oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc","kind":"zone"},"compression":{"type":"off"},"quota":null,"reservation":null},"dd9e2a12-9e1a-4c01-9090-2318a4ca5bf6":{"id":"dd9e2a12-9e1a-4c01-9090-2318a4ca5bf6","name":{"pool_name":"oxp_44152dd0-a9bc-40a0-a64f-d375b950e5bf","kind":"debug"},"compression":{"type":"gzip_n","level":9},"quota":107374182400,"reservation":null},"e517594e-729d-40dc-9b05-75f0866cc720":{"id":"e517594e-729d-40dc-9b05-75f0866cc720","name":{"pool_name":"oxp_15b53b30-72cf-4edb-a7c4-325ee3f7c679","kind":"external_dns"},"compression":{"type":"off"},"quota":null,"reservation":null},"eb609f8e-0bd4-4f07-8c8c-8ad72bc3b8e6":{"id":"eb609f8e-0bd4-4f07-8c8c-8ad72bc3b8e6","name":{"pool_name":"oxp_16b6a5fa-aa93-4781-aede-4393c243a1fc","kind":"debug"},"compression":{"type":"gzip_n","level":9},"quota":107374182400,"reservation":null}}}

This will all go away once we do implement deleting datasets (#6177 + #7304). In the meantime, we may want the planner to avoid creating this problem and I'll file a separate issue for that. But in case this does ever happen, Sled Agent really shouldn't allow this. As long as Sled Agent requires that the (pool name, kind) tuple be unique among its datasets, it should validate that up front in PUT /datasets and fail the request immediately, before having updated its local ledger.

@davepacheco
Copy link
Collaborator Author

Filed #7312 for the planner workaround.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant