Value Sources (Subsystem/Pipeline layer)

[KathleenDollard]

Working on default values and validation at nearly the same time, along with conversations with Chet about configuration of open ended completions, led to an understanding of unification in how we need to get values. This is currently in the codebase as ValueSource although the name is negotiable.

Value sources can be the raw value, the value with defaults, a calculated value, a value from an environment variable, a value from a configuration file, the default value for the type, or a calculated value based on each of these (independent), or a value calculated from whichever is found (final).

Default values

Work on default values shows that there is a clear fallback route that users will expect (this list is for defaults only):

• A user entered value
• A specified default value
• A calculcated value
• A value relative to another symbol's pipeline value (possibly with a calculation)
• A value from configuration (optionally with a calculation)
• A value from an environment variable (optionally with a calculation)
• The default value for the type

A specific symbol may have one or more of these (the default value for the type is always available).

The order of specified value and calculated value is arbitrary - it is illogical to include them both. The rest or ordered by nearest to farthest from the end user's perspective.

For user entered values, the value mechanism that follows this precedence should always be used. This is called the pipeline value in this document and is currently retrieved via `PipelineResult.GetValue().

Range validation

The bounds of range validation are just values that can be compared to the value being validated. These values can come from:

• A specified value
• A calculated value
• A value relative to another symbol's pipeline value (possibly with a calculation)
• A value from configuration (optionally with a calculation)
• A value from an environment variable (optionally with a calculation)
• An indication that the value is not found
  • Note: the current code does not account well for the case where the upper bound is an environment variable, if it is present and it is not (zero is used for the upper bound)

ValueSource

ValueSource expresses a source that may supply a value. In addition to the known sources, others can be imagined. For example, the .NET CLI may well create a custom value source for the current project filename, retrieved from exploring the current directory.

Expected value sources are:

• SimpleValueSource
• CalculatedValueSource
• RelativeToSymbolValueSource
• RelativeToConfigurationValueSource
  • Not yet written due to lack of standard, there maybe many
• RelativeToEnvironmentVariableValueSource

The use of "RelativeTo" in the naming is to indicate that a calculation can be provided.

Structuring these scenarios would allow documentation, for example, a display of all environment variables that could be set and how they are used.

Other uses

In addition to default values and validation, values may need to be retrieved to aid in running pipeline subsystems. The scenario that uncovered this is the number of values to return for completions in an open ended set (no upper bound), which could come from either an environment variable or configuration.

In addition to supplying values, I think it is important that value sources describe themselves. If a user is troubleshooting about a bad default value, or receives a message that an entered value can't be after 2024-06-29, they have few options. Walking through code may not be possible, and the fix (updating the environment variable) is in their circle of responsibility. It seems desirable for sources to describe themselves, at least in help, and possibly in validation diagnostics.

By far the most common kind of calculation for value sources will be adding or subtracting from values of the same type being returned. While this is trivial to write as a lambda, it is an allocation and much more importantly it loses descriptive content. Thus, it is desirable to let the user explicitly set an offset, rather than always using a lambda. This is straightforward for numbers, but will need some thoughts on dates where the offset will be an integer, rather than another date, and will be in one of multiple units, and will require calling methods rather than doing addition.

Descriptions

Value sources could have (localized) descriptions like the following:

• "42"
• Calculated values: must be entered by the CLI author
• "at least 7 days after hire date" where there is an option named --hire-date or --hireDate
• "at least 7 days after today (7 is from configuration: gracePeriod)"
• "at least 7 days after today (7 is from env variable: GRACE_PERIOD)"

Extension methods

Many, probably most, uses of value sources will be via extension methods. Thought will need to be given to how the conversion to a ValueSource is done. For example, if the extension method changed a number of days to a lambda expression, it would allocate for both the lambda expression and description, even if they were never used. For this reason, it is likely that we will have a significant number of value sources, although we need to ensure precedence remains correct.

Aggregating values

I greatly dislike the name "aggregating" here as that sounds like it collects rather than choosing the best of. "fallback" is a bit noun like, but perhaps is better. "Journey" is the way I think of it but seems an awful name. "Resolving"?

Ultimately, a value comes from a single source - if we need to support collections that are extended through aggregation, that is an entirely separate problem. Collections are supported in this proposal, but the entire collection comes from a single source. That said, it may be a side effect of the implementation proposed here that a collecting aggregate value source could be created.

There are three types of journeys to get a value. The most common, probably by far, will be a single ValueSource.

Next most common would be a value source in a couple levels of precedence, like the value the user entered, configuration and an environment variable.

The least common case is a single precedence level having more than one ValueSource. For example, a default value could conceivably 7 days before the value of one option if it is entered, but if it is not entered the default value should be 10 days before a different date.

An aggregating value source avoids complicating the common case for niche cases. This could be done with a AggregatingValueSource that had a collection of ValueSource it sorted based on precedence if it was needed. In the case of items with equal precedence, they can be called in the order entered.

Precedence order should be opinionated, but there may be additional levels of complexity for some scenarios. For example, multiple configuration files, possibly hierarchical via directory. Thus, precedence should be logically virtual, although nothing else in aggregate value source probably needs to be.

Calculated values

The design thus far means it is a baby step to allowing calculated values - although it is also feature creep. The benefits of calculated values:

• Code reuse if the same concept is used in multiple ways along with the ability to name it
  • For example: "EndOfProbabtion" which is 90 days after hire date may be the first date for certain actions
• Performance if an expensive value is used more than once
  • For example: the current project file may be used both in a must exist and a default value context
• Compound values
  • For example, cases like Point where a value may be created from several CliValueSymbols just works
  • For example, this could be an easy way to handle things like the dotnet nuget why problem where backwards compatibility requires the use of a first argument being optional
• Subsystems that need to define values for the Pipeline instance would have a simple place to store them with all the support described here
• Structuring these scenarios would allow documentation of calculated values and how they are calculated

The structure of a calculated value would be just a name and description, and the

This is a baby step because it could be implemented as:

• A new CalculatedValue class derived from CliValueSymbol
  • Pipeline code should use CliValueSymbol over CliArgument and CliOption
  • Because it is a CliValueSymbol extension methods, such as for setting validation just work
• A Pipeline.CalculatedValues collection for the definitions
• One more step in PipelineResult.GetValue(<CliValueSymol>) immediately after looking for the symbol that looks in Pipeline.CalculatedValues
  • If it is found, PipelineResult.GetValue it is used
• A dictionary Pipeline.CalculatedValuesByName because all name lookups are treated as a lookup in the current cone of execution to get the symbol, then normal use of the symbol
• Tests

This is a sufficiently small amount of work (after the rest), that I would recommend it even if only because it solves the nuget why problem so elegantly.

[KathleenDollard]

ValueSource.GetValue should return (success, value) (in lieu of a DU) to avoid blocking on the case where a calculation correctly returns null and no further processing should be done.