Package management using Lingo

[edwardalee]

I'm attempting use Lingo and create a reusable library following the instructions in this PR. My work is in this library. I have (too many) questions:

1. There is an mqtt-c in a separate lf-pkgs organization. Why a separate organization? I have no write permission in this organization, so I created an mqtt-c repo in lf-lang. Following this suggestion, I also created a pkgs repo in lf-lang with just one README file. It currently lists just this one library. Perhaps @vinzbarbuto could add his Python library here?
2. What is the difference between [lib] and [[app]] in Lingo? Why does [[app]] have double brackets? It seems that without a [lib] in the file, Lingo does not recognize a library, even though there is a lib directory. But it seems to not matter what [lib] points to in its main property. What should it point to?
3. Where should cmake files go in a library's directory tree? I've put them in src/include.
4. The instructions suggest there is just one Main.lf in src. Usually there will be multiple examples.
5. Why is [app.properties] a separate clause rather than just part of [[app]] in Lingo? Why is [app.properties] required? The examples show it setting fast = false, but this is a default property, so it seems unnecessary.
6. Why does lingo clean not remove the build directory? Also, perhaps it should remove the lfc-generated directories, bin, src-gen, fed-get, and
7. lingo build does not compile the generated code. It has no-compile: true. Why?
8. lingo build puts executables (though only for federated programs because of the no-compile) in build/bin. Shouldn't these go into bin to behave like lfc?
9. What is the lingo.lock file for? Should this me in a .gitignore?

[tanneberger]

1.) I like this suggestion. From my understanding, this would mean that we keep a list of packages in a machine-readable format that can be ingested by the vscode plugin and used to easily find packages. This would also lower the barrier of entry, because instead of moving a repo into an organization, we simply add an entry to this file.

2.) The difference between a [lib] and [[app]] is that a lib is a selection of lf files that a project exports as a library, while an [[app]] refers to an executable. The reason why [lib] has only one square bracket and [[app]] has two is due the fact that you can only specify one library in a Lingo.toml, but multiple executables. The main property should refer to the file where your main reactor is located.

3.) Technically, this doesn't matter, because within the Lingo.toml you use the cmake-include property to refer to this file. see: https://github.com/lf-pkgs/websocket-cpp/blob/main/Lingo.toml#L12

4.) You can have as many main reactors as you want, and for each executable program you create a [[app]] block in your Lingo.toml

5.) This is unfortunately just an effect of how the toml format works. I think [[app.properties]] is also more intuitive, but unfortunately not standard compliant. If you're interested, you can see the data structure behind this toml scheme: https://github.com/lf-lang/lingo/blob/main/src/package/mod.rs#L66

6.) Lingo puts all its build-artifacts in the build folder, so we can easily clean up the project by deleting just this one folder.

7.) Similar to reactor-uc, we treat lfc only as a code-generator, hence this flag is always set. The compilation and handling of the various target toolchains is completely handled by Lingo. Lingo has different "Backend" to build the different targets. Here is the backend that builds reactor-c programs: https://github.com/lf-lang/lingo/blob/main/src/backends/cmake_c.rs

8.) The binaries are also build-artifacts, so I also put them in the build folder.

9.) In technical jargon, one would say that the Lingo.lock is for pinning the dependencies. In the Lingo.lock, the packages are pinned exactly to a git-revision and sha1 hash. If a Lingo.lock is present Lingo, will just download what is specified in the lock-file, otherwise it needs to download and parse the whole dependency tree. I would advise against putting the Lingo.lock into your .gitignore.

[edwardalee]

2.) The difference between a [lib] and [[app]] is that a lib is a selection of lf files that a project exports as a library, while an [[app]] refers to an executable. The reason why [lib] has only one square bracket and [[app]] has two is due the fact that you can only specify one library in a Lingo.toml, but multiple executables. The main property should refer to the file where your main reactor is located.

A library doesn't need to have a main reactor and it may have more than one main reactor in demo files that are not really part of the library (not in the lib directory). The library itself could be in multiple files (very few libraries have just one book). So it's still not clear to me what to specify for the main property. Also, the brackets syntax seems pretty arbitrary. Why have brackets at all?

3.) Technically, this doesn't matter, because within the Lingo.toml you use the cmake-include property to refer to this file. see: https://github.com/lf-pkgs/websocket-cpp/blob/main/Lingo.toml#L12

Oh, interesting. What is the cmake-include directory relative to? Is this documented anywhere?

5.) This is unfortunately just an effect of how the toml format works. I think [[app.properties]] is also more intuitive, but unfortunately not standard compliant. If you're interested, you can see the data structure behind this toml scheme: https://github.com/lf-lang/lingo/blob/main/src/package/mod.rs#L66

Hmm. Maybe toml isn't a good choice for Lingo. JSON?

6.) Lingo puts all its build-artifacts in the build folder, so we can easily clean up the project by deleting just this one folder.

Should I file a bug report that lingo clean does not remove the build folder?

7.) Similar to reactor-uc, we treat lfc only as a code-generator, hence this flag is always set. The compilation and handling of the various target toolchains is completely handled by Lingo. Lingo has different "Backend" to build the different targets. Here is the backend that builds reactor-c programs: https://github.com/lf-lang/lingo/blob/main/src/backends/cmake_c.rs

How does one use this? I.e., how do I use Lingo to create binary executables? Again, is this documented anywhere?

9.) In technical jargon, one would say that the Lingo.lock is for pinning the dependencies. In the Lingo.lock, the packages are pinned exactly to a git-revision and sha1 hash. If a Lingo.lock is present Lingo, will just download what is specified in the lock-file, otherwise it needs to download and parse the whole dependency tree. I would advise against putting the Lingo.lock into your .gitignore.

How does one do the pinning? Where is the syntax of Lingo.lock defined? The file that appears in my directory is an empty file, so it's not clear what that one does.

[tanneberger]

Okay let me give you a brief overview of the toml format:

[package]
name = "example_project"
version = "0.1.0"
authors = ["tassilo.tannerber@tu-dresden.de"]
homepage = "https://lf-lang.org"
license = "Weird Stallman License"
description = "A little Lingo.toml for people"

The section [package] is used to declare meta information about the current package.

[lib]
name = "websocket"
location = "./src/lib"
target = "C"
platform = "Native"

[lib.properties]
cmake-include="./websocket.cmake"

Each package can export exactly one library, which is specified in the [lib] section. The most important value here is the location attribute, which should refer to a folder containing the files that are to be made available to the users of this library. Libraries can also have additional properties ([lib.properties]) for example the "cmake-include" property. When Lingo compiles an executable, it collects all these properties of the libraries used and composes them together.

[[app]]
name = "git-hook"
target = "Cpp"
main = "src/Main.lf"

[[app.properties]]
cmake-include = "./my-cmake.cmake"
logging = "info"

App blocks are used to define executables in your Lingo.toml. Your package can export multiple [[apps]]. The most important property in an [[app]] block is the main property, which must point to the file that contains the main reactor for this app.

Is this documented anywhere?

No, unfortunately I haven't written any documentation for the toml schema yet.

The format of the files "Lingo.toml" and "Lingo.lock" is defined by the Rust data structures, from which the parsers are (mostly) automatically generated.

• Lingo.toml: https://github.com/lf-lang/lingo/blob/main/src/package/mod.rs#L66-L80
• Lingo.lock: https://github.com/lf-lang/lingo/blob/main/src/package/lock.rs#L186-L194