Sample Go application with database backend

Yes, this is one more sample app as there are probably thousands already on the internet. We have tons of sample apps from different vendors with various types of licenses available on different repositories. Nevertheless sometimes I struggle to find exactly what I need – a simple app with a database backend which can work with Oracle Autonomous databases and optionally with Postgres backend. In my everyday life I primarily use Go as a programming language and I would like to have such an app written using that language. So, eventually I gave up and created my own application with a simple frontend and two (as for now) options for backend databases – Oracle and Postgres.

Continue reading “Sample Go application with database backend”

Oracle Apex on ATP with custom domain

If you have an Apex app on top of your Oracle Autonomous database you have the application URL like https://m5c5hpup7eqqydh-glebatp01.adb.us-ashburn-1.oraclecloudapps.com/ords/r/covid/covid-ontario/covid_ontario. This is already better than it used to be before and have a friendly path in the URL but what if you want to use your own domain address and custom URL? This blog is about how to set it up using reverse proxy. The way with a custom ORDS is a subject for another post.

So, you have your own domain registered in DNS, for example apex.gleb.ca and an application like I’ve listed above. What you want to do is to make the application URL as https://www.apex.gleb.ca/covid/ . What we are going to use is a virtual machine with a Nginx web server serving as a reverse proxy to your Apex application created on the Oracle Autonomous database.

The first step is to get the virtual machine for the proxy. In the Oracle Cloud you have an “Always Free” tier which includes a couple of AMD based VMs and the ability to create up to four ARM instances free of charge. You can read more about all free resources here. I’ve chosen an ARM type machine VM.Standard.A1.Flex with one CPU and 4 GB of memory and Oracle Linux 8.6. It will be quite enough to run a small proxy instance. I am not describing the provisions of the instance or how to subscribe to the Oracle OCI assuming that if you already have an Oracle Apex application on Autonomous you know that. But if you are starting from scratch here is a good place to start.
Having the instance and connected as the opc user now it is time to install the required software.

sudo dnf update
sudo dnf install nginx

When it is installed we enable, start and verify the nginx service on the VM.

sudo systemctl start nginx
sudo systemctl status nginx

The next step is to make the http and https open in the firewall.

firewall-cmd --permanent --zone=public --add-service=http
firewall-cmd --permanent --zone=public --add-service=https
firewall-cmd --reload

Then we can prepare a network security group with open 80 and 443 ports to the VM instance. If you are on OCI you are going to “Networking” -> “Virtual Cloud Network” -> your network, click on your network and click on the “Network Security Groups” on the right. Here you can add a new group and provide ingress access to port 80 and 443. Here is an example.

And assign the security group to our proxy VM.

Now if you try to put your domain name such as apex.gleb.ca or the public IP for the instance to the address bar in the browser you should see the default Nginx welcome page.

Now we need to configure our application proxy in the Nginx configuration. The configuration files are located in the /etc/nginx directory. We are creating a new configuration file for our application in the /etc/nginx/conf.d directory where we define the main parameters. Here is an initial configuration.

#Default folder
server {
    listen         80;
    listen         [::]:80;
    server_name    apex.gleb.ca www.apex.gleb.ca;
    root           /usr/share/nginx/html/apex.gleb.ca;
    index          index.html;
    try_files $uri /index.html;
}

Now we can copy the index.html to the new directory and test it in our browser. I’ve slightly modified the title in the file to see the difference. You can see the “on apex.gleb.ca” in the first line.

sudo mkdir  /usr/share/nginx/html/apex.gleb.ca
sudo cp /usr/share/nginx/html/index.html /usr/share/nginx/html/apex.gleb.ca/
sudo nginx -t
vi /usr/share/nginx/html/apex.gleb.ca/index.html

Now we need to get a certificate for our domain if you still don’t have one. I am using “Let’s Encrypt” (https://letsencrypt.org/) for my domains including my blog site. Here is how I’ve done that for the apex.gleb.ca:

pip3 install certbot
pip3 install certbot-nginx
certbot --nginx
 
#output
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Python 3.6 support will be dropped in the next release of Certbot - please upgrade your Python version.
Enter email address (used for urgent renewal and security notices)
 (Enter 'c' to cancel): glebot@hotmail.com
 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017-w-v1.3-notice.pdf.
You must agree in order to register with the ACME server. Do you agree?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: Y
 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing, once your first certificate is successfully issued, to
share your email address with the Electronic Frontier Foundation, a founding
partner of the Let's Encrypt project and the non-profit organization that
develops Certbot? We'd like to send you email about our work encrypting the web,
EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: N
Account registered.
 
Which names would you like to activate HTTPS for?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: apex.gleb.ca
2: www.apex.gleb.ca
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel): 1,2
Requesting a certificate for apex.gleb.ca and www.apex.gleb.ca
 
Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/apex.gleb.ca/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/apex.gleb.ca/privkey.pem
This certificate expires on 2022-12-03.
These files will be updated when the certificate renews.
 
Deploying certificate
Successfully deployed certificate for apex.gleb.ca to /etc/nginx/conf.d/apex.gleb.ca.conf
Successfully deployed certificate for www.apex.gleb.ca to /etc/nginx/conf.d/apex.gleb.ca.conf
Congratulations! You have successfully enabled HTTPS on https://apex.gleb.ca and https://www.apex.gleb.ca

If you have a look at your Nginx configuration file apex.gleb.ca.conf you will notice new lines added by the certbot.

    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/apex.gleb.ca/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/apex.gleb.ca/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

Having SSL setup now we can correct our configuration for Nginx and define a proxy for the Apex application. We need to add a couple of sections there to define our location and the image directory. The final configuration file will look like this:

#Default folder
server {
    server_name    apex.gleb.ca www.apex.gleb.ca;
    root           /usr/share/nginx/html/apex.gleb.ca;
    index          index.html;
    try_files $uri /index.html;
 
 
  location /covid {
    proxy_pass https://m5c5hpup7eqqydh-glebatp01.adb.us-ashburn-1.oraclecloudapps.com/ords/r/covid/covid-ontario/covid_ontario;
    proxy_set_header Origin "" ;
    proxy_set_header X-Forwarded-Host $host:$server_port;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_connect_timeout       600;
    proxy_send_timeout          600;
    proxy_read_timeout          600;
    send_timeout                600;
  }
 
  location /i/ {
    proxy_pass https://m5c5hpup7eqqydh-glebatp01.adb.us-ashburn-1.oraclecloudapps.com/i/;
    proxy_set_header X-Forwarded-Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  }
 
    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/apex.gleb.ca/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/apex.gleb.ca/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
 
 
}
 
server {
    if ($host = www.apex.gleb.ca) {
        return 301 https://$host$request_uri;
    } # managed by Certbot
 
 
    if ($host = apex.gleb.ca) {
        return 301 https://$host$request_uri;
    } # managed by Certbot
 
 
    listen         80;
    listen         [::]:80;
    server_name    apex.gleb.ca www.apex.gleb.ca;
    return 404; # managed by Certbot
 
 
}

The final things are to modify SELinux parameters to allow forward requests and restart the Nginx.

sudo setsebool -P httpd_can_network_relay 1
sudo getsebool -a | grep httpd
sudo systemctl start nginx

Now our site should be reachable using our custom URL.

I hope this blog can help to make the first steps. Of course you can improve the architecture by placing a load balancer in front of Nginx proxy and making it highly available. Also this configuration might cause some problems with Cross-Origin Resource Sharing (CORS) info and it can be solved by installing and registering the ORDS in our Autonomous database. But this is the subject for another blog.

Oracle Database Service for Azure

Recently Oracle and Microsoft announced availability of their new service – Oracle Database Service for Azure. It raises the level of integration and interoperability between Oracle Cloud and Microsoft Azure. With the new service Microsoft’s customers are able to create and manage Oracle databases created in Oracle cloud and use them for applications located in Azure. That sounds great and moves us one step closer to a real multi-cloud environment. Before proceeding, let me clarify what I mean as the “real” multi-cloud. I mean an environment where different pieces of the same IT service are located in different clouds working as a whole.

Continue reading “Oracle Database Service for Azure”

MongoDB on Oracle Autonomous

Earlier this year Oracle announced support for MongoDB API on the Oracle Autonomous database family. It has been recognized as a huge milestone by Forbes and multiple other analysts, technical and market experts. In my opinion it was logical and fully predicted one more step to the main idea of “Converged Database”. I didn’t coin that term – Oracle was using it for two or three years as of now. The main idea is to create a fully managed database platform supporting the most of APIs and interfaces and put the data together under the Oracle Autonomous Database umbrella. Here I will try to look inside and understand what it is, how it works, what is supported and what is not.

Continue reading “MongoDB on Oracle Autonomous”

Data Pump Export on Oracle Autonomous

The blog was supposed to be a small how-to but it has grown to a bigger one and hopefully might help to avoid some minor problems while exporting data from an Autonomous Database (ADB) in Oracle Cloud (OCI). It is about exporting data to the Oracle DataPump format to move data to another database or as a logical “backup”.

Oracle documentation provides sufficient information but I find it more and more difficult to navigate considering the number of options and flavours for Oracle databases. There are some new ways and tools around Oracle OCI ATP which can help in some cases. If you want you can jump directly to the end to read the summary.

Continue reading “Data Pump Export on Oracle Autonomous”

Blog on Oracle OCI ARM free tier.

This blog was primarily driven by questions from my peers and colleagues who wondered where my blog was hosted and how it was created. It might help to move from a hosting platform to your own website and where to start.

Like most bloggers I started my blog using one of the hosting platforms but soon after found some limitations in choosing appearance, plugins and was a bit annoyed by some commercial banners on my page. After a while I decided to move to my own site. I bought a domain name for myself and created my own environment using the WordPress software on a cloud VM. It didn’t cost me too much but it was not entirely for free. When Oracle introduced some additions to the always free set of resources. I decided to give it a try and move my blog entirely to the OCI free tier.

For those who would like to skip the reading and try, I have a set of Terraform scripts on GitHub. They haven’t been updated lately and use not the latest versions but can be a good place to start.

Continue reading “Blog on Oracle OCI ARM free tier.”

Terraform modules simplified.

Terraform is probably already the de-facto standard for cloud deployment. I use it on a daily basis deploying and destroying my tests and demo setups in my Oracle cloud tenancy. Sometimes the deployment environment for a demo has too many files or some files inside are really big and hard to read due to the number of different resources and parameters included there. How can we make our configuration more usable? Let’s try Terraform modules and demonstrate how they work.
For our tests we are going to use terraform v1.0.3 and Oracle Cloud Infrastructure (OCI). You will need a working OCI and on your machine with terraform defined environment variables. The full list of required environment variables will be provided in the README file in the GitHub repository.
Let’s say we have a simple demo or test configuration with a dedicated network, internet gateway and a VM. And we want to assign multiple security rules using security lists and maybe one or two security groups. We can include all those rules to the configuration file for the network but maybe there is a better way. What if we want to reuse the similar set of the security rules and security groups not only to that deployment but share with some other stacks? We can try to use Terraform modules.

Continue reading “Terraform modules simplified.”

Exadata Cloud at Customer – free space in ASM and adding a new database.

Some time ago I wrote a short blog about dependencies between the number of enabled CPUs and how many databases you could build. Today we got another error when we were trying to create a new database. Here is the screenshot of the error.

If you can’t read it on a small screen it says “Create Database operation failed due to an unknown error. Refer to work request ID 2580d3ff-064e-4e6f-ab06-1327fd02f40e when opening a Service Request at My Oracle Support.” and provide an error code which is “Error

Continue reading “Exadata Cloud at Customer – free space in ASM and adding a new database.”

Linux Hugepages and AUTO_ONLY in Oracle 19c.

Most Oracle DBA are sufficiently educated about benefits using large memory pages for Oracle database SGA to reduce overhead and improve performance. If you want to read more about it you can start from that Oracle blog or read it from other multiple articles and blogs. Oracle is using parameter use_large_pages to direct behaviour of an Oracle instance during startup.

In the previous versions before 19c we had three possible values – “TRUE”, “FALSE and “ONLY”. Since Oracle 11.2.0.3 the “TRUE” meant that the instance will allocate as many hugepages as free available in the system and get the rest from the normal small pages. The “FALSE” would tell it to not use the hugepages at all and the “ONLY” would be able to start an instance only if sufficient number of free hugepages is available in the system to fit all SGA in it. The “TRUE” was default for all databases. 

In the 19c version we got one more value – “AUTO_ONLY” and now it is the default value for Exadata systems running Oracle Database 19c. The description in documentation is not totally clear and sounds very similar to the description of “ONLY” value. Here is an excerpt from the documentation:

“It specifies that, during startup, the instance will calculate and request the number of large pages it requires. If the operating system can fulfill this request, then the instance will start successfully. If the operating system cannot fulfill this request, then the instance will fail to start.”

Let me show you how it works. Here is my sandbox with a 19c database and no hugepages is configured on the box by default. 

Continue reading “Linux Hugepages and AUTO_ONLY in Oracle 19c.”

Google Bare Metal in numbers.

In the previous posts I shared my first impression and how to start using the Google Bare Metal Service (BMS). In this post I will try to show some numbers related to the performance of the solution and you can compare it with your existing environment.

Let me start from the box characteristics. For my tests I was using a “o2-standard-32-metal” box located in the us-west2 zone (Los Angeles) . The solution was configured with 2Gbps interconnect and had a couple of storage resources attached to it. The first one was represented by two 512Gb disks based on HDD storage where I placed my binaries and a recovery ASM disk group and the second was a 2Tb volume “all flash” I used for data.  Here is summary table:

Characteristic
BMS Box typeo2-standard-32-metal
CPU Intel(R) Xeon(R) Gold 6234 CPU @ 3.30GHz
CPU sockets2
CPU cores16
Memory384 GB
Disk 1512 Gb – Standard disk
Disk 2 512 Gb – Standard disk
Disk 3 2048 Gb – All flash
Network4 NICs Speed: 25000Mb/s
OS Oracle Linux 7.9
BMS box characteristics.

Before starting the tests I updated my Oracle Linux and installed a number of packages required for my Oracle database and packages to test IO and Network such as fio and iperf3. Here is a summary table with software and tools used to test the performance.

PackageTesting scope
fioIO performance
stress-ngCPU. Memory
swingbenchOracle database performance
SLOBOracle database IO
iperf3Network
oratcptestNetwork
Continue reading “Google Bare Metal in numbers.”