withParam parsing of JSON/YAML lists

[masonlr]

Is this a BUG REPORT or FEATURE REQUEST?: BUG REPORT

What happened:

Submitting a workflow with the --parameter-file flag led to workflow failure with the error message

withParam value could not be parsed as a JSON list: [map[message:foo] map[message:another]]

What you expected to happen:

--parameter-file flag should allow for list structures in both YAML and JSON files.

How to reproduce it (as minimally and precisely as possible):

1. Create a Workflow in task-loop-from-list.yaml:

# task-loop-from-list.yaml
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: loops-
spec:
  entrypoint: loop-example
  arguments:
    parameters:
    - name: messages
      value: |
        [
          { "message": "foo" },
          { "message": "bar" },
          { "message": "baz" }
        ]

  templates:
  - name: loop-example
    inputs:
      parameters:
      - name: messages
    steps:
    - - name: print-message
        template: whalesay
        arguments:
          parameters:
          - name: message
            value: "{{item.message}}"
        withParam: "{{inputs.parameters.messages}}"

  - name: whalesay
    inputs:
      parameters:
      - name: message
    container:
      image: docker/whalesay:latest
      command: [cowsay]
      args: ["{{inputs.parameters.message}}"]

2. Confirm submission with no parameters:

$ argo submit task-loop-from-list.yaml
Name:                loops-blpgp
Namespace:           default
ServiceAccount:      default
Status:              Pending
Created:             Sun Jun 02 17:45:30 +0100 (now)
Parameters:
  messages:          [
  { "message": "foo" },
  { "message": "bar" },
  { "message": "baz" }
]

3. Confirm submission with command line parameters:

$ argo submit task-loop-from-list.yaml -p messages='[{"message": "another"}, {"message": "foo"}]'
Name:                loops-lscd9
Namespace:           default
ServiceAccount:      default
Status:              Pending
Created:             Sun Jun 02 17:46:25 +0100 (now)
Parameters:
  messages:          [{"message": "another"}, {"message": "foo"}]

4. Attempt submission with --parameter-file flag:

$ cat params.json
{
    "messages": [{"message": "foo"}, {"message": "another"}]
}

$ argo submit task-loop-from-list.yaml --parameter-file params.json
Name:                loops-h6w7q
Namespace:           default
ServiceAccount:      default
Status:              Pending
Created:             Sun Jun 02 17:48:31 +0100 (now)
Parameters:
  messages:          [map[message:foo] map[message:another]]

$ cat params.yaml
messages:
- message: foo
- message: another

$ argo submit task-loop-from-list.yaml --parameter-file params.yaml
Name:                loops-mj7wd
Namespace:           default
ServiceAccount:      default
Status:              Pending
Created:             Sun Jun 02 17:49:04 +0100 (now)
Parameters:
  messages:          [map[message:foo] map[message:another]]

Anything else we need to know?:

Environment:

• Argo version:

$ argo version
argo: v2.2.1
  BuildDate: 2018-10-11T16:25:59Z
  GitCommit: 3b52b26190163d1f72f3aef1a39f9f291378dafb
  GitTreeState: clean
  GitTag: v2.2.1
  GoVersion: go1.10.3
  Compiler: gc
  Platform: darwin/amd64

• Kubernetes version :

$ kubectl version -o yaml
clientVersion:
  buildDate: "2019-03-26T00:04:52Z"
  compiler: gc
  gitCommit: 641856db18352033a0d96dbc99153fa3b27298e5
  gitTreeState: clean
  gitVersion: v1.14.0
  goVersion: go1.12.1
  major: "1"
  minor: "14"
  platform: darwin/amd64
serverVersion:
  buildDate: "2018-11-26T14:25:46Z"
  compiler: gc
  gitCommit: 637c7e288581ee40ab4ca210618a89a555b6e7e9
  gitTreeState: clean
  gitVersion: v1.10.11
  goVersion: go1.9.3
  major: "1"
  minor: "10"
  platform: linux/amd64

Other debugging information (if applicable):

• workflow result:

$ argo get  loops-h6w7q
Name:                loops-h6w7q
Namespace:           default
ServiceAccount:      default
Status:              Failed
Message:             withParam value could not be parsed as a JSON list: [map[message:foo] map[message:another]]
Created:             Sun Jun 02 17:48:31 +0100 (2 minutes ago)
Started:             Sun Jun 02 17:48:31 +0100 (2 minutes ago)
Finished:            Sun Jun 02 17:48:31 +0100 (2 minutes ago)
Duration:            0 seconds
Parameters:
  messages:          [map[message:foo] map[message:another]]

STEP            PODNAME  DURATION  MESSAGE
 ✖ loops-h6w7q                     withParam value could not be parsed as a JSON list: [map[message:foo] map[message:another]]

$ argo get loops-mj7wd
Name:                loops-mj7wd
Namespace:           default
ServiceAccount:      default
Status:              Failed
Message:             withParam value could not be parsed as a JSON list: [map[message:foo] map[message:another]]
Created:             Sun Jun 02 17:49:04 +0100 (1 minute ago)
Started:             Sun Jun 02 17:49:04 +0100 (1 minute ago)
Finished:            Sun Jun 02 17:49:04 +0100 (1 minute ago)
Duration:            0 seconds
Parameters:
  messages:          [map[message:foo] map[message:another]]

STEP            PODNAME  DURATION  MESSAGE
 ✖ loops-mj7wd                     withParam value could not be parsed as a JSON list: [map[message:foo] map[message:another]]

[ian-howell]

I've been able to replicate. Fortunately, I've also been able to fix it: #1397