Distributed Shell

Old: Project Goals

• "Toil" for multi-cloud distributed builds: http://www.oilshell.org/blog/2020/11/fixes-and-updates.html#buildssrht-and-toil. Problems:
  • Does too much work (not incremental), and doesn't do it fast enough (not parallel)
    • needs dependencies for both problems!
  • In some cases, the framework has more overhead than the work done by the application. We want lightweight distributed processes.
  • YAML is a really bad syntax for a shell script.
• Pash and Posh are related: https://github.com/oilshell/oil/issues/867

Notes on gg (Ad Hoc Multi-Cloud Distribution with Lambdas)

• Initial reaction: https://lobste.rs/s/virbxa/papers_i_love_gg#c_nbmnod
• More reactions
  • I really like the framing: low latency (which is why I use shell in the first place), warm vs. cold clusters
• Concepts
  • Model Substitution
  • Tail Calls
  • Dynamic dependendencies, not static (how does it relate to Shake?)
  • Lambdas can talk to each other (via NAT traversal?) Solves a well known performance issue.
• Citations
  • UCop
  • Ciel