Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-56740

Artifacts bigger than 5GB can't be uploaded to S3

    Details

    • Similar Issues:

      Description

      Unfortunately S3 have 5GB limit for single operation upload object: https://docs.aws.amazon.com/AmazonS3/latest/dev/UploadingObjects.html

      • Upload objects in a single operation—With a single PUT operation, you can upload objects up to 5 GB in size.
      • Upload objects in parts—Using the multipart upload API, you can upload large objects, up to 5 TB.

      This plugin don't implement multipart upload and can't upload big files to S3.

        Attachments

          Issue Links

            Activity

            Hide
            dstanhope D Stanhope added a comment -

            I quite like this plugin, and have also run into this issue.
            For now, I have coded a PowerShell script I can call like a function, using Write-S3Object (Which auto-detects file size and uses multi-part upload as necessary) to upload a file or folder.
            If this is useful to others I'm happy to post it here.

            Show
            dstanhope D Stanhope added a comment - I quite like this plugin, and have also run into this issue. For now, I have coded a PowerShell script I can call like a function, using Write-S3Object (Which auto-detects file size and uses multi-part upload as necessary) to upload a file or folder. If this is useful to others I'm happy to post it here.
            Hide
            bcxpro Carlos Barcenilla added a comment -

            D Stanhope Could you share the details of your PowerShell workaround?

            Show
            bcxpro Carlos Barcenilla added a comment - D Stanhope Could you share the details of your PowerShell workaround?
            Hide
            dstanhope D Stanhope added a comment -

            Here's a generalised version:

            Copy-ToAwsS3.psm1

            Documentation is in the file.
            Note that you'll probably want to find a different way of handling credentials than what that file implements, or at least switch to using PowerShell Secure Strings

            Hope that helps!

            Show
            dstanhope D Stanhope added a comment - Here's a generalised version: Copy-ToAwsS3.psm1 Documentation is in the file. Note that you'll probably want to find a different way of handling credentials than what that file implements, or at least switch to using PowerShell Secure Strings Hope that helps!

              People

              • Assignee:
                bozaro Artem V. Navrotskiy
                Reporter:
                bozaro Artem V. Navrotskiy
              • Votes:
                2 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: