This blog post is bit longer than usual but I wanted to cover at least three options to upload files to the Oracle OCI object storage. If you need to just upload one file you can stop reading after the first option since it covers probably most of needs to upload a single file. But if you want a bit more it makes sense to check other options too.
The OCI Object storage has a web interface with an “Upload object” button, but sometimes you need to upload files directly from a host where you have only a command line shell. In general we have at least three ways how to do that.
The first and the simplest way is to create a temporary “Pre-Authenticated Request” which will expire after specified time. The procedure is easy and intuitive.
You need to go to your bucket details and click on the right side to open the tab with “Pre-Authenticated Requests”
Push the button “Create Pre-Authenticated Request”, choose the name and expiration time for the link.
The link will appear in a pop up window only once, and you have to copy and save it if you want to use it later. If you have forgotten to do that it is not a problem – you can create another one.
I’ve created a link used it to upload a test file to the “TestUpload” bucket without any problem.
[opc@sandbox tmp]$dd if=/dev/zero of=random_file.out bs=1024k count=5 5+0 records in 5+0 records out 5242880 bytes transferred in 0.001785 secs (2937122019 bytes/sec) [opc@sandbox tmp]$ll total 10240 -rw-r--r-- 1 otochkin staff 5.0M 9 Mar 09:55 random_file.out [opc@sandbox tmp]$curl -T random_file.out https://objectstorage.ca-toronto-1.oraclecloud.com/p/PCmrR1tN3D_5SkJimndiatnClEwNQbnMpaVHfYYwio4/n/gleb/b/TestUpload/o/ [opc@sandbox tmp]$It is the easiest way but what if you want to set up a more permanent process without the disappearing links. Maybe the uploading is going to be a part of data flow or you want to schedule a regular activity. The answers are the Oracle OCI CLI and the Rest API interface using API keys. Let’s check how we can do it without installing Oracle OCI CLI.
The first thing you need is an “API key”. Behind the scenes it is a public part of a secret key you create on your box where you plan too run your scripts or in your application.
[opc@sandbox ~]$ mkdir ~/.oci [opc@sandbox ~]$ openssl genrsa -out ~/.oci/oci_api_key.pem 2048 [opc@sandbox ~]$ chmod go-rwx ~/.oci/oci_api_key.pem [opc@sandbox ~]$ openssl rsa -pubout -in ~/.oci/oci_api_key.pem -out ~/.oci/oci_api_key_public.pem [opc@sandbox ~]$ cat ~/.oci/oci_api_key_public.pem -----BEGIN PUBLIC KEY----- MIOFIjANBg..... ... cQIDYQAB -----END PUBLIC KEY----- [opc@gleb-bastion-us ~]$You need to copy and paste the output from the last command (from “—–BEGIN PUBLIC KEY” to “—–END PUBLIC KEY—–“) to the form which appears when you push the button “Add Public Key” on your user details.
Having the API key in your profile on OCI we can now use the function oci-curl provided by oracle and use it in our command line. But before doing that we need to gather some values to pass to the function. The tenancy id can be found in your tenancy details you can get from the drop-down menu in top right conner of your OCI webpage. The same menu provides your user details where we need the user id. The key fingerprint for our recently created key can be found on the same page.
Now you can change the section in the script replacing the OCID by your values
# TODO: update these values to your own local tenancyId="ocid1.tenancy.oc1..aaaaaaaaba3pv6wkcr4jqae5f15p2b2m2yt2j6rx32uzr4h25vqstifsfdsq"; local authUserId="ocid1.user.oc1..aaaaaaaat5nvwcna5j6aqzjcaty5eqbb6qt2jvpkanghtgdaqedqw3rynjq"; local keyFingerprint="20:3b:97:13:55:1c:5b:0d:d3:37:d8:50:4e:c5:3a:34"; local privateKeyPath="/Users/someuser/.oci/oci_api_key.pem";Instead of fixing the OCID in the script you may choose to use environment variables providing them either by an “export” command in the shell or putting them to a file with environments. Here is example how you can do that.Creating a file[opc@sandbox ~]$ vi .oci_env privateKeyPath=~/.oci/oci_api_key.pem keyFingerprint="c9:3b:97:13:55:1c:5b:0d:d3:37:d8:50:4e:c5:3a:fe" authUserId=ocid1.user.oc1..aaaaaaaaaba3pv6wkcr4jqae5f15p2b2m2yt2j6rx32uzr4h25vqstifsfdsq tenancyId=ocid1.tenancy.oc1..aaaaaaaat5nvwcna5j6aqzjcaty5eqbb6qt2jvpkanghtgdaqedqw3rynjq compartmentId=ocid1.compartment.oc1..aaaaaaaa4laqzjcaty5eqbb6qt7cdfx2jl4d7bvuitvlmz4b5c2hiz6dbssza endpoint=objectstorage.ca-toronto-1.oraclecloud.com namespace=mytenancyname bucketName=TestUpload export privateKeyPath keyFingerprint authUserId tenancyId compartmentId endpoint namespace bucketNameYou can see that in addition to the OCID in the script I’ve added endpoint, namespace, bucket name and OCID for my compartment. Those values we need to upload our files. We can use the file to export all those variables.[opc@sandbox ~]$ source .oci_env [opc@sandbox ~]$Download the signing_sample_bash.txt, remove the lines with values for OCID and paths, and remove the the UTF-8 Byte Order Mark from the file replacing it by a simple “#” symbol.[opc@sandbox ~]$ curl -O https://docs.cloud.oracle.com/iaas/Content/Resources/Assets/signing_sample_bash.txt % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 4764 100 4764 0 0 8707 0 --:--:-- --:--:-- --:--:-- 8709 [opc@sandbox ~]$ sed -i "/\(local tenancyId=\|local authUserId=\|local keyFingerprint=\|local privateKeyPath=\)/d" signing_sample_bash.txt [opc@sandbox ~]$ file signing_sample_bash.txt signing_sample_bash.txt: UTF-8 Unicode (with BOM) text [opc@sandbox ~]$ sed -i "1s/^.*#/#/" signing_sample_bash.txt [opc@sandbox ~]$ file signing_sample_bash.txt signing_sample_bash.txt: ASCII text [opc@sandbox ~]$Run the script.[opc@sandbox ~]$ source signing_sample_bash.txt [opc@sandbox ~]$Now we can use the “oci-curl” function in our command line and upload files to an OCI bucket without installing software to the machine.Create a file.[opc@sandbox ~]$ dd if=/dev/urandom of=new_random_file.out bs=1024k count=10 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.188255 s, 55.7 MB/s [opc@sandbox ~]$Upload from the command line[opc@sandbox ~]$ oci-curl $endpoint put ./new_random_file.out /n/$namespace/b/$bucketName/o/new_random_file.out [opc@gleb-bastion-us ~]$We can list the files:[opc@sandbox ~]$ oci-curl $endpoint get /n/$namespace/b/$bucketName/o/{"objects":[{"name":"another_random_file.out"},{"name":"new_random_file.out"}]} [opc@sandbox ~]$And we can see the fileYou can see more examples how to use the oci-curl function in the Oracle blog.The last way is to install the Oracle OCI CLI as it is described in documentation. It will take only few minutes. You need to run just one command and answer to some questions.[opc@sandbox ~]$ bash -c "$(curl -L https://raw.githubusercontent.com/oracle/oci-cli/master/scripts/install/install.sh)" % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 6283 100 6283 0 0 23755 0 --:--:-- --:--:-- --:--:-- 23889 Downloading Oracle Cloud Infrastructure CLI install script from https://raw.githubusercontent.com/oracle/oci-cli/6dc61e3b5fd2781c5afff2decb532c24969fa6bf/scripts/install/install.py to /tmp/oci_cli_install_tmp_mwll. ######################################################################## 100.0% Python3 not found on system PATH Running install script. ... output was reduced.Then you need to configure the CLI using “oci setup config” command.[opc@sandbox ~]$ oci setup configThis command provides a walkthrough of creating a valid CLI config file
…It will ask about tenancy and use OCID and suggest to create new keys but you can say “n” if you have already the key.... Enter a region (e.g. ca-toronto-1, eu-frankfurt-1, uk-london-1, us-ashburn-1, us-gov-ashburn-1, us-gov-chicago-1, us-gov-phoenix-1, us-langley-1, us-luke-1, us-phoenix-1): ca-toronto-1 Do you want to generate a new RSA key pair? (If you decline you will be asked to supply the path to an existing key.) [Y/n]: n Enter the location of your private key file: /home/opc/.oci/oci_api_key.pem Fingerprint: 20:3b:97:13:55:1c:5b:0d:d3:37:d8:50:4e:c5:3a:34 Config written to /home/opc/.oci/config If you haven't already uploaded your public key through the console, follow the instructions on the page linked below in the section 'How to upload the public key': https://docs.us-phoenix-1.oraclecloud.com/Content/API/Concepts/apisigningkey.htm#How2 [opc@sandbox ~]$And we can use the oci command line interface to upload or list the files or perform other actions.
[opc@sandbox~]$ oci os object put -bn TestUpload --file one_more_random_file.out Uploading object [####################################] 100% { "etag": "31a3ae0c-5749-4390-8bae-d937a1709d9a", "last-modified": "Wed, 03 Apr 2019 16:21:48 GMT", "opc-content-md5": "s18Q1y1YYX113hBOqA19Mw==" } [opc@sandbox ~]$ oci os object list -bn TestUpload { "data": [ { "md5": "y3wX2q+fN+lBHppGMJqfhw==", "name": "another_random_file.out", "size": 5242880, "time-created": "2019-03-10T16:16:33.707000+00:00" }, { "md5": "/XHj/5+IkyoDbLteg6E/7w==", "name": "new_random_file.out", "size": 10485760, "time-created": "2019-04-03T15:05:47.270000+00:00" }, { "md5": "s18Q1y1YYX113hBOqA19Mw==", "name": "one_more_random_file.out", "size": 10485760, "time-created": "2019-04-03T16:21:47.734000+00:00" } ], "prefixes": [] } [opc@sandbox ~]$As a short summary I want to say that the oci-cli command line interface can be useful and provides easy way for regular operations when the REST API can be extremely useful when you want to incorporate it to your code and use in some of your applications or cannot install any tools on your box due to some restrictions.
