load.Instances with string literal CUE content

[dghubble]

Is your feature request related to a problem? Please describe.

I'd like to use the cue Go API to load cue content from both literal content and paths, but the API seems unclear if it supports this.

# foo.cue
{
  name: config.name
}

instances := load.Instances("foo.cue", config)

literal := `
  config: {
    name: "bar"
  }
`

cuectx := cuecontext.New()
value1 := cuectx.CompileString(content)
value2, _ := cuectx.BuildInstances(instances)

// how to build literal and instances at the same time? or get the literal to be an "instance"?
// desired output
{
  name: "bar"
}

With literal CUE contents, it seems like about all the API let's us do is convert them directly to a Value. Whereas in this example, we want to "evaluate" them all together. The load.Instances(args, config) takes string filenames, whereas most Go APIs would offer an io.Reader that would let use provide the content to be loaded in a variety of ways.

Overall, the terms context, instance, build, compile, and load make it very unclear how to use the Go API. I've gathered that to "build" cue stuff together you need to have all the bits of config be available. Values are for "fully rendered" stuff, you can't take the string content value and the BuildInstances Value and merge them together.

Is it possible to combine expressions that originate from string literals and files? Or am I required to write them out to files?

[dghubble]

tldr: I've got a bunch of CUE contents, some are in Go strings some are files on disk. How do I use this Go library?

[myitcv]

Hi @dghubble - I've moved your issue to be a discussion in the Q&A category.

For an example of how to load CUE via the Go API please see https://cuelang.org/docs/tutorial/loading-cue-go-api/.

If you wanted to augment what is on disk with more data, you could use an Overlay as part of a https://pkg.go.dev/cuelang.org/go/cue/load#Config value that is passed to load.Instances.

Would that cover your use case?

Happy to create an example of this for cuelang.org if that would be of help.

[myitcv]

I've created cue-lang/docs-and-content#149 to capture the creation of new content for cuelang.org either way. cc @jpluscplusm

[dghubble]

I'm not sure I follow how one would use this API. I have some cue expressions as string literals and others in files, and I just want to evaluate ("build"?) them all together. I was expecting an API that takes io.Reader's or something idiomatic.

For a concrete example, its removing the limitation in https://github.com/poseidon/terraform-provider-cue/blob/bc647ff27b5e873348a414445d7f2db851c69442/internal/data_config.go#L75 that only allows CUE expressions to be passed as string literals OR filenames, not both.

[myitcv]

I'm not sure I follow how one would use this API. I have some cue expressions as string literals and others in files, and I just want to evaluate ("build"?) them all together.

When CUE loads code, it can operate at two levels: standalone files or as packages:

exec cue export .:z x.cue y.cue
cmp stdout stdout.golden

-- x.cue --
x: 3
-- y.cue --
y: 1
-- z1.cue --
package z

z1: 3
-- z2.cue --
package z

z2: z1
-- stdout.golden --
{
    "x": 3,
    "y": 1,
    "z1": 3,
    "z2": 3
}

Notice how with package values you can have references between files that belong to the same package: with standalone files (x.cue and y.cue in the example above) you cannot.

Relating this back to the code you linked to: if someone provides content and paths, how should it be combined?

I think the following code gives a hint:

https://github.com/poseidon/terraform-provider-cue/blob/bc647ff27b5e873348a414445d7f2db851c69442/internal/data_config.go#L157

i.e. it suggests that every path and hence perhaps every piece of content should just be unified together. Whether this makes sense in the context of terraform-provider-cue I will defer to you!

But that linked code provides the answer: the way you combine cue.Value values is through cue.Value.Unify. That linked code is using a cue.Context to build instances into cue.Value values and then unify (or combine) them together. You could also use cue.Context.CompileString to create a cue.Value from your content and unify that with the result from loadPaths. Again, whether it makes sense or not to do this is dependent on the application context.

A passing comment about this observation:

// Cue's API is buggy. Unifying with a zero value doesn't work.

This is captured in #3175, but as a workaround you can use:

ctx := cuecontext.New()
v := ctx.CompileString("_")

[dghubble]

Thanks for the suggestions.

it suggests that every path and hence perhaps every piece of content should just be unified together

That was my first attempt. To use cuectx.BuildInstances(instances) to create a value from path-based files. And cuectx.CompileString(content) to create a value from raw content, and then unify them. But the path-based files need to refer to config.name, where the config is defined via the string content.

# foo.cue
package example
{
  name: config.name
}

content := `
  package example
  config: {
    name: "bar"
  }
`

with package values you can have references between files that belong to the same package: with standalone files you cannot.

That might be the trouble. Effectively, I'm looking for a way to build a value, using the path-based or string content expressions, as though they're all part of the same package. Using BuildInstances doesn't seem to work, I think it wants to build a concrete value right then and there. _|_ // reference "config" not found. It doesn't seem to have a way of partially constructing a value.

So then I'm looking for a way to pass cuectx.BuildInstances both the path-based and string expressions together.

EDIT: I did find a way to use the Overlay option to get BuildInstances to "load" the string expression as though it were a file. poseidon/terraform-provider-cue#48