Add S3 guard to prevent publishing existing jars

[Ben-El]

Closes #9387

This PR strengthens the deployment workflow to prevent overwriting released Spark client jars
on the treeverse-clients-us-east bucket.

Changes

• Added a safe publish step (s3PutIfAbsent) that uses If-None-Match: *, ensuring S3 rejects uploads if the object already exists.
• Updated publish-scala to run s3PutIfAbsent.
• Removed the non-performant recursive aws s3 cp --acl public-read ... step.
• Updated readme (clients/spark/README.md).

Public readability right now is applied by adding "--acl","public-read" flag addition to the s3PutIfAbsent command, but in the near future it will be enforced at the bucket level, not via ACLs.

Testing

As elaborated in the issue discussion.

Verified:

• New jars upload successfully.
• Re-publishing an existing version failed as expected.
• Jars remain publicly accessible (because of the "--acl","public-read" flag addition in the s3PutIfAbsent command).

Follow ups:

• In S3, on treeverse-clients-us-east bucket - switch Object Ownership to Bucket owner enforced (disable acl's), to make objects public via bucket policy.
• Remove the "--acl","public-read" flag addition in the s3PutIfAbsent command
  (Remove public acl flag from s3PutIfAbsent task in spark/build.sbt #9512).

[nopcoder]

if we run aws s3 cli to perform the upload, we need to update the requirements from the env / readme.
also, sbt as a build tool should be able to run command, we can use it - without adding code to compile in order to run a command.

[nopcoder]

search for non 's3api' command - aws s3 is preferred.

[Ben-El]

@nopcoder

Using aws s3 cp instead of s3api put-object is possible with some "overhead tweak", but there are trade-offs:

aws s3 cp can perform multipart uploads, however, it does not support the If-None-Match: "*" condition. That means you cannot guarantee an atomic “create-if-absent” operation with aws s3 cp.

If we switch to aws s3 cp, the only workaround would be something like -

• checking first: aws s3api head-object --bucket "$bucket" --key "$key" ....
• And then uploading: aws s3 cp "$jar" "s3://$bucket/$key"....

But this is not atomic.

For our use case, since the assembly jar is always < 5GB, I think s3api put-object would fit nicely (it supports single-object uploads up to 5GB).

If one day we publish artifacts larger than 5GB, we would need multipart upload (and lose the atomic “no-overwrite” guarantee).

[nopcoder]

lgtm

[Isan-Rivkin]

Please release experimental version from branch after merge for final sanity check?
Verify its accesible from S3 with no auth or aws cli, just curl and that's the jar works!

[Ben-El]

Please release experimental version from branch after merge for final sanity check? Verify its accesible from S3 with no auth or aws cli, just curl and that's the jar works!

Works fine !