Salesforce, Python, SQL, & other ways to put your data where you need it

Need event music? 🎸

Live and recorded jazz, pop, and meditative music for your virtual conference / Zoom wedding / yoga class / private party with quality sound and a smooth technical experience

💬 EN

#

Is there anything I can do to ephemerally spin up and configure an Oracle database with just a few tables and rows per table, in a way that will be performant enough to run in a CI/CD pipeline

Yes, you can ephemerally spin up and configure an Oracle database with a few tables and rows to use in a CI/CD pipeline. Here are some approaches:

  • Use Docker Containers: Oracle provides Docker images for databases. Pre-create a seed database with your tables and data in a persistent volume or baked into the image. This allows quick container startup, ensuring consistent environments for tests1.
  • Pluggable Databases (PDBs): Use Oracle’s pluggable database feature to clone a lightweight PDB for testing. This is faster than creating a new database from scratch and provides isolation12.
  • SQLcl and Liquibase: Automate schema and data provisioning using tools like SQLcl and Liquibase. These tools integrate well with CI/CD pipelines for applying changes deterministically34.
  • Data Virtualization: Use Oracle’s CloneDB or third-party virtualization tools to quickly create test instances from snapshots of an existing database12.

These methods ensure performance and reliability while fitting into CI/CD workflows.

⁂

Where would I store the image definition to get the fastest spin-up, if my testing runs in AWS CodeBuild off of code stored in AWS CodeCommit?

To achieve the fastest spin-up time for your Oracle database image in AWS CodeBuild with code stored in AWS CodeCommit, you should store the image definition in Amazon Elastic Container Registry (ECR). Here’s why:

  • Proximity to CodeBuild: ECR is fully integrated with AWS services, including CodeBuild, which minimizes latency during image pulls.
  • Performance: ECR is designed for high availability and low-latency access, ensuring quick container startup times.
  • Integration: CodeBuild can directly authenticate and pull images from ECR without additional configuration.
  • Cost Efficiency: ECR charges only for storage and data transfer, making it cost-effective compared to third-party registries.

You can push your Docker image to ECR and reference it in your CodeBuild project configuration for seamless integration.

⁂

How fast could the pipeline probably run, then?

The pipeline’s runtime will depend on several factors, but with proper optimization, you can achieve a fast execution time. Here’s an estimate based on typical configurations:

  1. Source Retrieval: Pulling code from AWS CodeCommit typically takes a few seconds, as AWS has optimized this process to be very fast (e.g., ~7 seconds reported recently)5.
  2. Container Spin-Up: Using an Oracle database image stored in Amazon ECR, container spin-up time can range from 10 to 30 seconds, depending on the image size and caching67.
  3. Build and Test Execution: Running lightweight builds and tests with caching enabled (e.g., for dependencies or Docker layers) can take 1–3 minutes in AWS CodeBuild78. For small databases with a few tables, database initialization should add minimal overhead.
  4. Parallelization: If steps like database setup and application tests are parallelized, the overall runtime could be reduced further96.

Estimated Total Time: With optimizations, your pipeline could run in approximately 2–5 minutes for lightweight builds and tests.

⁂

  1. https://oracle-base.com/articles/misc/oracle-databases-and-cicd-pipelines ↩ ↩2 ↩3

  2. https://www.oracle.com/a/ocom/docs/database/implementing-devops-principles-with-oracle-database.pdf ↩ ↩2

  3. https://blogs.oracle.com/database/post/part-1-oracle-database-cicd-with-sqlcl-and-liquibase ↩

  4. https://www.reddit.com/r/oracle/comments/o2kyd8/cicd_pipeline_for_oracle_19c_database/ ↩

  5. https://www.reddit.com/r/aws/comments/1bz4cz6/improve_cicd_pipeline_to_decrease_build_time/ ↩

  6. https://www.nclouds.com/blog/accelerate-cicd-pipeline-aws-codepipeline-codebuild/ ↩ ↩2

  7. https://docs.aws.amazon.com/codebuild/latest/userguide/build-caching.html ↩ ↩2

  8. https://stackoverflow.com/questions/70544643/is-there-any-way-to-speed-up-aws-code-build-with-maven ↩

  9. https://cloud.folio3.com/blog/optimizing-aws-codepipeline-workflows/ ↩