Marathon run causes the update of all cached Packages

[lf-araujo]

Hi all,

Whenever I start a new project I have manually remove cached libraries, because as soon as I run marathon run script it causes an attempt to update all installed Packages. This then results in two problems (I am running marathon on linux):

1. Marathon fails if the internet is not good enough, as it pings remote github addresses;
2. If any of the old packages have been changed and for some reason don't compile anymore I cannot run the current script, regardless if this script uses those packages or not.

Thank you for this software.

Edit 1: I momentarily closed this issue because it is a bit trick to reproduce. But here is a MWE that happens every time. If one wants to make a nice little command line interface program with marathon, the best option on linux is Console, which is part of Vapor. Something in the update marathon attempts to do, makes anything built with Console only build correctly for the first time. Copy the code below:

import Console // marathon: https://github.com/vapor/console.git

let terminal = Terminal()
print(terminal is Console) // true
terminal.print("Hello")

/// Outputs the prompt then requests input.
let name = terminal.ask("What is your name?")
terminal.print("You said: \(name)")

if terminal.confirm("Are you sure?") {
    // they are sure
} else {
    // don't do it!
}

This will work in the first attempt, but in the second one will be greeted by a bug during the update/rebuilt of the package:

  �[0;3m$ cd "/home/luis/.marathon/Scripts/Cache/-home-luis-Dropbox-Documentos-Coding-Swift-Programs-CSVImporter-example-example/" && swift build -C /home/luis/.marathon/Scripts/Cache/-home-luis-Dropbox-Documentos-Coding-Swift-Programs-CSVImporter-example-example/ �[0;23m
   �[0;3mwarning: '--chdir/-C' option is deprecated; use '--package-path' instead
   warning: PackageDescription API v3 is deprecated and will be removed in the future; used by package(s): CSwiftV
   'example' /home/luis/.marathon/Scripts/Cache/-home-luis-Dropbox-Documentos-Coding-Swift-Programs-CSVImporter-example-example: error: product dependency 'swift-nio' not found
   'example' /home/luis/.marathon/Scripts/Cache/-home-luis-Dropbox-Documentos-Coding-Swift-Programs-CSVImporter-example-example: error: product dependency 'swift-nio-zlib-support' not found�[0;23m

[bitwit]

I think I'm seeing a similar issue in my own project right now. It seems to be pulling in and adding all the subdependencies as dependencies.

For example, I was using PathKit, which uses Spectre to do testing. After running a couple of times I start getting complaints about XCTest not being available (which Spectre uses). Spectre has nothing to do with PathKit other than during tests though.

Hoping to investigate this more myself too. I suspect it is related to the way pinned packages work, but can not yet confirm that. Marathon attempts to avoid duplicate dependencies from different scripts, but I wonder if something in either the Swift language or Marathon has changed recently that is causing this.

[bitwit]

Okay, I can confirm the problem is consistent with mine and provide some more info now. It's all about what's going on in the cache. I'll try to keep it short and to the point:

• Marathon caches a bunch of data inside of ~/.marathon.
• Marathon creates a "master" at ~/.marathon/Packages/Generated/Package.swift file that it uses as the template for generating the Package.swift file of each respective script which live at locations using this pattern: ~/.marathon/-Users-name-and-path-to-script-slug-script-name/Package.swift
• After updating twice, all dependencies and their children get listed in the Generated/Package.swift which is used as a template.

See evidence below. I used your terminal.swift code in the example for consistency.

~/.marathon/Packages/Generated/Package.swift after initial install of terminal.swift

~/.marathon/...terminal/Package.swift after initial install

~/.marathon/Packages/Generated/Package.swift after marathon update

~/.marathon/...terminal/Package.swift after update/run

If you have multiple scripts with dependencies, their dependencies likely start to intermingle also, and all future compilations fail. It also could increase the compilation time of very basic scripts.

Seems to me that the template Package.swift pattern probably needs revising. I'm currently investigating support for multiple dependencies from the same Package, which has been surfacing this problem too. See #191