DEV Community: Nabhaas Cloud Consulting

Series Week 20 / 52 — Differentiating Patching differences between Exadata On Prem and OCI Databases (DB Systems and ExaCS)

Abhilash Kumar | Oracle ACE ♠ — Sun, 05 Apr 2026 06:45:44 +0000

{ Abhilash Kumar Bhattaram : Follow on LinkedIn }

Moving on from the previous week 19 ( from my previous blog )

https://dev.to/nabhaas/week-19-52-oracle-databases-choosing-between-on-prem-or-any-cloud-oci-aws-gcp-azure--5b0p

While understanding the ExaCS vs Exadata On Prem , we are trying to comprehend the patching differences to manage Exadata On Prem vs ExaCS , a cloud environment is still a war when compared to On Prem nevertheless its still the DBA's capability the rules to manage these machines.

Some useful refresher for Oracle On Prem - Exadata Patching

https://blogs.oracle.com/exadata/exadata-mos-notes
https://www.oracle.com/a/tech/docs/exadata-software-maintenance-2022.pdf

Some useful refresher for Oracle OCI - ExaCS Patching

https://docs.oracle.com/en-us/iaas/exadatacloud/doc/ecs-patch-update.html

A quick tabular differences of On Prem Patching and OCI Class of databases


-- On Prem
+----------------------+---------------------------+
| Component            | On-Prem Exadata           | 
+----------------------+---------------------------+-
| Cell Patching        | Quarterly (Manual)        | 
| Infiniband Patching  | Quarterly (Manual)        | 
| Dom0 Patching        | Not Applicable            | 
| DomU VM Patching     | Not Applicable            | 
| Grid Infrastructure  | Quarterly (Manual)        | 
| Database Patching    | Quarterly (Manual)        | 
+----------------------+---------------------------+


-- OCI ExaCS 
+----------------------+---------------------------+----------------------------------------------------------------+
| Component            | On-Prem Exadata           | ExaCS (OCI)                                                    |
+----------------------+---------------------------+----------------------------------------------------------------+
| Cell Patching        | Quarterly (Manual)        | Managed by Oracle (Implicit) as a part of Dom0 auto schedules  |
| Infiniband Patching  | Quarterly (Manual)        | Managed by Oracle (Implicit) as a part of Dom0 auto schedules  |
| OS Patching          | DB Node Patching (Manual) | Quarterly (Customer Driven)                                    |
| Grid Infrastructure  | Quarterly (Manual)        | Quarterly (Customer Driven)                                    |
| Database Patching    | Quarterly (Manual)        | Quarterly (Customer Driven)                                    |
+----------------------+---------------------------+----------------------------------------------------------------+


-- OCI ExaCS 
+----------------------+---------------------------+----------------------------------------------------------------+
| Component            | On-Prem Exadata           | DB System (OCI)                                                |
+----------------------+---------------------------+----------------------------------------------------------------+
| Cell Patching        | Not Applicable            | Not Applicable                                                 |
| Infiniband Patching  | Not Applicable            | Not Applicable                                                 |
| OS Patching          | OS specific               | Bundled with Grid Patcing                                      |
| Grid Infrastructure  | Quarterly (Manual)        | Quarterly (Customer Driven)                                    |
| Database Patching    | Quarterly (Manual)        | Quarterly (Customer Driven)                                    |
+----------------------+---------------------------+----------------------------------------------------------------+

1. Ground Zero: Where Challenges Start

----------------------------------------------------------------------------------------------------
## 1. Ground Zero: Where Challenges Start - Understand Patching Problems
----------------------------------------------------------------------------------------------------


- Massive Infrastructure Footprint: You are managing multiple ExaCS VM Clusters scattered 
across different OCI compartments, each with its own lifecycle and governance needs.

- Application Silos: Your databases are logically segregated to support 
diverse application stacks. This isn't just one database; it’s an entire fleet 
of distinct environments.

- The Scale Paradox: With dozens of databases across multiple clusters, the sheer volume 
of "moving parts" makes manual oversight impossible. You aren't just a DBA anymore; 
you are a Fleet Manager.

2. Underneath Ground Zero:

----------------------------------------------------------------------------------------------------
## 2. Underneath Ground Zero: Finding the Real Problem ( problem of plenty )
----------------------------------------------------------------------------------------------------


- The Maintenance Collision: Between Dom0 (Hypervisor), DomU (Guest VM), 
Grid Infrastructure, and the Database layer, the sheer number of required updates 
(for both Prod and Non-Prod) far exceeds the number of available maintenance windows.

- The Downtime Crunch: Business units are demanding 24/7 availability. 
Finding a slot to take down a cluster for a rolling patch is a high-stakes 
negotiation with stakeholders.

- The Certification Bottleneck: Applications are stubborn. Some require specific 
DB versions (e.g., 19.18 vs 19.21) for certification. 
You can’t just "patch everything to latest"  without breaking the application tier.

- Version Fragmentation: You need the surgical ability to maintain multiple database 
versions on the same infrastructure to satisfy different application compatibility 
requirements without compromising the security of the whole cluster.

3. Working Upwards:


----------------------------------------------------------------------------------------------------
## 3. Working Upwards: From Understanding to Solution - Simplify and Automate
----------------------------------------------------------------------------------------------------

From the above it is evident you need the following 

- Patching automation 
- Multiple and Flexbile patching windows 
- Logical Patching windows ( e,g. patch all GI in a window and DB in another window )
- Logical Patching methods ( patch Dom-U VM , GI , DB ) in that order for every cluster  

Now you get to see the patching conundrum 

TAB from Nabhaas does this for you 

- The Orchestration Engine: TAB (Total Automation Box) isn't just a process; 
it’s an automation framework designed to handle the "heavy lifting" of 
ExaCS Databases lifecycle management.

- Strategic Pre-checks: TAB runs exhaustive, automated health checks before the first 
line of code is patched. If the cluster isn’t healthy, TAB stops the process before the 
downtime starts.

- Parallel Fleet Patching: While manual patching is linear (one by one), 
TAB allows for parallelized, orchestrated updates across multiple VM clusters, 
drastically reducing the total "time-to-compliance."

- Version Pinning & Flexibility: TAB gives the CTO the power to patch 
the Grid Infrastructure  to the latest secure version while "pinning" specific 
Databases to the exact versions required by application owners.

- Predictable Rollbacks: In the rare event of a patch failure, TAB provides a structured, 
rollback path, ensuring that "failed patching" doesn't turn into "extended outage."

- Governance at Scale: TAB provides a single pane of glass for patching status across all 
compartments, transforming a chaotic spreadsheet exercise into a predictable, 
audit-ready dashboard.

Eventually you can see that OCI Patching is an orbital shift to the traditional way of Exadata Patching.

How Nabhaas helps you

If you’ve made it this far, you already sense there’s a better way — in fact, you have a way ahead.

If you’d like Nabhaas to assist in your journey, remember — TAB is just one piece. Our Managed Delivery Service ensures your Oracle operations run smoothly between patch cycles, maintaining predictability and control across your environments.

TAB - Whitepaper ,
download here

Managed Delivery Services - Whitepaper ,
download here

Series Week 19 / 52 — Oracle Databases choosing between On Prem or any Cloud [ OCI , AWS , GCP , Azure ]

Abhilash Kumar | Oracle ACE ♠ — Fri, 27 Mar 2026 05:08:40 +0000

{ Abhilash Kumar Bhattaram : Follow on LinkedIn }

Oracle database can be hosted in all major cloud vendors [ OCI , AWS , GCP , Azure ] , there is a lot of reference material here.
https://www.oracle.com/in/cloud/multicloud/

This blog is about going through the decision making process of a CTO which cloud or hybrid setup to use

Organisations already have Exadata Infra and are looking to upgrade their infra
Organizations that are trying to implement new Exadata and are considering cloud solutions
Organizations that are into AWS, Azure , GCP trying to make best use of Oracle databases

1. Ground Zero: Where Challenges Start


> Organizations already running Exadata (On-Prem)
  Many are approaching a hardware refresh cycle. While the system still works,
  questions around cost, utilization, and long-term sustainability begin to surface.
  - The pressure to “move to cloud” often starts here — sometimes without full clarity.
  - There are regulatrory requirements to stay On Prem


> Organizations planning new Exadata adoption
  These teams are at the beginning of their journey, trying to balance a large upfront
  investment against cloud-based flexibility. In most cases, workload sizing is still
  unclear, making decisions heavily influenced by vendor narratives.

> Organizations already in AWS / Azure / GCP
  Here, Oracle databases are often running on non-native platforms, leading to
  compromises in performance, licensing, or scalability. OCI enters the conversation
  as a specialized option — but only for the database layer.

> Organizations evaluating Autonomous vs ExaCS
  There is growing curiosity around Autonomous Database, often seen as the “future.”
  At the same time, there is hesitation — especially where applications require control,
  customization, or certified environments.

>> Different starting points, but all leading to the same question:
   What is the right platform and service model for our databases?

2. Underneath Ground Zero: Finding the Real Problem

> Existing Exadata Customers
  The decision is often framed too narrowly — as a choice between upgrading hardware
  or moving to the cloud. What is usually missing is a deeper look at how the system
  is actually being used, and what it truly costs to operate.

> New Exadata Adopters
  Without clear workload insights, there is a tendency to overestimate requirements.
  This leads to over-sizing, higher costs, and unnecessary complexity from day one.

> Multi-Cloud Organizations
  OCI is sometimes treated as just another cloud provider, rather than a platform
  optimized specifically for Oracle databases. This leads to gaps in network design,
  identity integration, and overall architecture.
  Organizations use heavy applications native to AWS , Azure , GCP , 
  the egress cost would be a silent killer with multi cloud [ unless you have a dedicated bandwidth provider ]

> Autonomous vs ExaCS Decision Gap
  Autonomous is often assumed to be the default choice, but not all applications fit.
  Enterprise systems like EBS or heavily customized databases may require capabilities
  that Autonomous intentionally abstracts away.

> The Common Thread
  Across all these scenarios, decisions are driven more by trends and assumptions
  than by actual workload behavior and application needs.

>> The real challenge is understanding the service model behind it.

3. Working Upwards: From Understanding to Solution


> For Existing Exadata Customers
  If workloads are stable and predictable, continuing on-prem with an optimized
  refresh strategy can still make sense. However, if demand fluctuates or data center
  overhead becomes a concern, ExaCS offers a more flexible path forward.

> For New Exadata Adopters
  The focus should be on understanding the workload first — usage patterns, peak
  demand, and I/O behavior. Starting small and scaling gradually is often more
  effective than committing to a large footprint upfront.

> For Multi-Cloud Organizations
  OCI should be positioned thoughtfully — not as a generic cloud, but as a
  database-focused platform. Network design, identity alignment, and cost
  considerations need to be handled deliberately.

> Autonomous vs ExaCS — Making the Right Call

  Autonomous Database works best when:
    - Workloads are standardized
    - Operational overhead needs to be minimized
    - Full automation is acceptable

  Autonomous Database does not work best when:

    - Application stack version compatibility needs to be done regularly due to complex app stack.


  ExaCS is more suitable when:
    - Fine-grained control over the database is required
    - Applications have specific certification or dependency needs
    - Custom configurations and integrations are critical

Bringing It All Together

The most effective decisions come from aligning three things:
- The nature of the workload
- The level of control required
- The operational model the organization is ready for

CTO Outcome : Instead of forcing a single direction, the goal is to place each workload on the platform where it performs best — with clarity, control, and predictability.

The right choice depends on the application and the service model , the challenge is to understand the goal and identify the means.

How Nabhaas helps you

If you’ve made it this far, you already sense there’s a better way — in fact, you have a way ahead.

TAB - Whitepaper ,
download here

Managed Delivery Services - Whitepaper ,
download here

Series Week 18 / 52 — Managed Cost Optimization: Controlling OCI Database Spends

Abhilash Kumar | Oracle ACE ♠ — Sun, 08 Mar 2026 16:30:20 +0000

{ Abhilash Kumar Bhattaram : Follow on LinkedIn }

Moving on to a common topic of costs , an oversight on operations can dent a hole in Cloud databse spends.

The promise of the cloud—predictable costs and seamless elasticity—often clashes with the reality of OCI-hosted Oracle environments. When costs spiral, the culprit is rarely the platform itself; rather, it's an architecture that lacks built-in cost governance.

Expensive inefficiencies usually stem from three common oversights:

Static Provisioning: Databases are sized for "worst-case" peak loads and left that way indefinitely.

Idle Resources: Non-production environments remain active 24/7, burning budget while teams are offline.

Data Bloat: Backups, archivelogs, and stagnant data accumulate silently without a lifecycle review.

True cost optimization in OCI isn't a post-mortem finance meeting; it is a continuous operational discipline. By shifting from passive consumption to deliberate management, a CTO can transform OCI from a source of budget surprises into one of the most transparent and controllable assets in the enterprise portfolio.

1. Ground Zero: Where Challenges Start

+--------------------------------------------------------------------------------------------------+
| 1. Ground Zero: Where Challenges Start                                                           |
+--------------------------------------------------------------------------------------------------+
| - Databases sized for peak load but running idle most of the time                                |
| - Non-production environments running 24x7                                                       |
| - Over-provisioned Exadata or DB Systems                                                         |
| - Storage growth not monitored strategically                                                     |
| - Multiple small databases instead of consolidation                                              |
| - Cloud costs reviewed only during budget cycles                                                 |
|                                                                                                  |
| Typical Symptoms                                                                                 |
| • Development environments cost nearly as much as production                                     |
| • Storage consumption grows without clear ownership                                              |
| • Compute capacity rarely adjusted after initial deployment                                      |
| • Test environments forgotten after project completion                                           |
|                                                                                                  |
| >> Cloud makes scaling easy.                                                                     |
|    But without governance, it also makes waste easy.                                             |
+--------------------------------------------------------------------------------------------------+

2. Underneath Ground Zero:



+--------------------------------------------------------------------------------------------------+
| 2. Underneath Ground Zero: Finding Where OCI Costs Can Be Controlled                             |
+--------------------------------------------------------------------------------------------------+
|                                                                                                  |
| Compute Optimization                                                                             |
| • Right-sizing OCPU allocations for DB Systems                                                   |
| • Scaling ExaCS compute nodes only when required                                                 |
| • Using burstable or smaller shapes for non-prod workloads                                       |
|                                                                                                  |
| Storage Optimization                                                                             |
| • Monitoring ASM diskgroup utilization                                                           |
| • Removing unused tablespaces and orphaned datafiles                                             |
| • Archivelog retention optimization                                                              |
|                                                                                                  |
| Environment Lifecycle Management                                                                 |
| • Scheduling non-prod shutdowns (nights/weekends)                                                |
| • Decommissioning unused environments                                                            |
| • Refreshing non-prod with smaller configurations                                                |
|                                                                                                  |
| Consolidation Opportunities                                                                      |
| • Multiple schemas instead of multiple databases                                                 |
| • PDB-based consolidation in multitenant environments                                            |
| • Shared infrastructure for low-volume workloads                                                 |
|                                                                                                  |
| Data Growth Control                                                                              |
| • Partition lifecycle management                                                                 |
| • Archiving historical data                                                                      |
| • Purging unused application logs                                                                |
|                                                                                                  |
| Licensing Awareness                                                                              |
| • Aligning OCPU counts with license entitlements                                                 |
| • Avoiding accidental over-allocation of licensed cores                                          |
|                                                                                                  |
| Backup and DR Strategy                                                                           |
| • RMAN retention policy tuning                                                                   |
| • Evaluating DR environment sizing                                                               |
| • Storage tier optimization for backups                                                          |
|                                                                                                  |
| Monitoring and Visibility                                                                        |
| • Cost tracking by environment and business unit                                                 |
| • Tagging resources for financial accountability                                                 |
|                                                                                                  |
| >> Cost optimization is rarely one big decision.                                                 |
|    It is dozens of small architectural choices.                                                  |
+--------------------------------------------------------------------------------------------------+

3. Working Upwards:

+--------------------------------------------------------------------------------------------------+
| 3. Working Upwards: From Understanding to Solution                                               |
+--------------------------------------------------------------------------------------------------+
| - Establish cost visibility dashboards for OCI database services                                 |
| - Align environment sizing with real workload patterns                                           |
| - Implement scheduled shutdown for non-prod environments                                         |
| - Consolidate low-utilization databases where appropriate                                        |
| - Introduce periodic storage and archivelog audits                                               |
| - Align DR architecture with realistic recovery objectives                                       |
| - Integrate cost review into operational governance                                              |
|                                                                                                  |
| Mature OCI Cost Governance Model                                                                 |
| • Quarterly architecture reviews                                                                 |
| • Automated non-prod lifecycle controls                                                          |
| • Storage growth alerts                                                                          |
| • Business-unit cost accountability                                                              |
|                                                                                                  |
| CTO Outcome                                                                                      |
| • Predictable cloud spending                                                                     |
| • Infrastructure aligned with business value                                                     |
| • Reduced waste without reducing capability                                                      |
|                                                                                                  |
| >> Cloud efficiency is not about spending less.                                                  |
|    It is about spending intentionally.                                                           |
+--------------------------------------------------------------------------------------------------+

How Nabhaas helps you

If you’ve made it this far, you already sense there’s a better way — in fact, you have a way ahead.

TAB - Whitepaper ,
download here

Managed Delivery Services - Whitepaper ,
download here

Series Week 17 / 52 — Benchmarking Oracle Service KPIs: Uptime, MTTR, MTBF

Abhilash Kumar | Oracle ACE ♠ — Fri, 27 Feb 2026 07:53:08 +0000

{ Abhilash Kumar Bhattaram : Follow on LinkedIn }

Moving on to the hot topic on on Clouds for enterprises - Uptime , MTTR and MTBF

Cloud providers advertise 99.99% or 99.999% availability, but very few teams pause to understand what is actually being promised. Availability SLAs are defined at the service infrastructure layer — not necessarily at the business service layer. The SLA may cover compute instance availability, storage durability, or control plane uptime — but it does not automatically include application responsiveness, database performance degradation, misconfiguration, architectural saturation, or customer-managed failover delays. The devil is in the definitions: what counts as downtime, what is excluded (planned maintenance, regional events, customer actions), and how availability is measured.

MTTR ( Mean Time to Repair ) - For OCI Databases ( DB Systems and ExaCS ) , this is customer defined For Autonomous Databases is this OCI defined

Therefor MTBF ( Mean Time Between Failures ) - is a maturity indicator.

ExaCS HA Documentation here

ADB HA Documentation here

1. Ground Zero: Where Challenges Start


+--------------------------------------------------------------------------------------------------+
| 1. Ground Zero: Where Challenges Start                                                           |
+--------------------------------------------------------------------------------------------------+
| - Uptime reported as “database was open”                                                         |
| - No distinction between planned vs unplanned downtime                                           |
| - MTTR calculated inconsistently                                                                 |
| - Incident severity not standardized                                                             |
| - No historical trend tracking                                                                   |
|                                                                                                  |
| Typical KPI Mistakes                                                                             |
| • Counting partial outages as “available”                                                        |
| • Ignoring performance degradation in uptime calculation                                         |
| • Measuring MTTR from ticket assignment, not incident start                                      |
| • No clear incident start/end timestamp discipline                                               |
| • DR failovers counted as uptime improvement without context                                     |
|                                                                                                  |
| >> Numbers exist. Reliability insight does not.                                                  |
+--------------------------------------------------------------------------------------------------+

2. Underneath Ground Zero:

+--------------------------------------------------------------------------------------------------+
| 2. Underneath Ground Zero: Finding the Real Problem                                              |
+--------------------------------------------------------------------------------------------------+
| - SLAs defined commercially, not operationally                                                   |
| - Monitoring tools not aligned with business impact                                              |
| - No separation between infrastructure and application outage                                    |
| - Root cause trends not tied to KPI analysis                                                     |
| - No MTBF tracking to identify systemic instability                                              |
|                                                                                                  |
| Hidden Structural Issues                                                                         |
| • Frequent small incidents masking instability                                                   |
| • Reactive firefighting improves MTTR but not MTBF                                               |
| • Patching windows inflating downtime metrics                                                    |
| • Global teams measuring differently                                                             |
| • Lack of service-level ownership                                                                |
|                                                                                                  |
| Core Reality                                                                                     |
| You cannot improve what you don’t measure correctly.                                             |
| And you cannot measure correctly without consistent definitions.                                 |
+--------------------------------------------------------------------------------------------------+

3. Working Upwards:

+--------------------------------------------------------------------------------------------------+
| 3. Working Upwards: From Understanding to Solution                                               |
+--------------------------------------------------------------------------------------------------+
| - Define precise KPI definitions                                                                 |
|   • Uptime = Service availability excluding approved maintenance                                 |
|   • MTTR = Time from incident detection to full service restoration                              |
|   • MTBF = Total uptime / number of unplanned failures                                           |
|                                                                                                  |
| - Standardize incident classification & severity models                                          |
| - Automate incident timestamp capture                                                            |
| - Separate performance degradation from full outages                                             |
| - Track rolling 30/90/180 day KPI trends                                                         |
| - Correlate KPIs with change events (patches, deployments, upgrades)                             |
| - Benchmark across environments (Prod vs DR vs Non-Prod)                                         |
|                                                                                                  |
| Mature Oracle Service KPI Model                                                                  |
| • Uptime tied to business service, not instance status                                           |
| • MTTR trending downward through structured response                                             |
| • MTBF increasing as architectural stability improves                                            |
| • KPI dashboards shared with leadership                                                          |
|                                                                                                  |
| CTO Outcome                                                                                      |
| • Data-driven reliability decisions                                                              |
| • Justified architecture investments                                                             |
| • Predictable SLA adherence                                                                      |
| • Fewer executive surprises                                                                      |
|                                                                                                  |
| >> Uptime is a result.                                                                           |
|    MTTR is a capability.                                                                         |
|    MTBF is a maturity indicator.                                                                 | 
+--------------------------------------------------------------------------------------------------+

How Nabhaas helps you

If you’ve made it this far, you already sense there’s a better way — in fact, you have a way ahead.

TAB - Whitepaper ,
download here

Managed Delivery Services - Whitepaper ,
download here

Series Week 16 / 52 — Automating Prechecks & Postchecks: Managed Services vs DIY

Abhilash Kumar | Oracle ACE ♠ — Mon, 16 Feb 2026 10:04:03 +0000

{ Abhilash Kumar Bhattaram : Follow on LinkedIn }

We have talked about how to automated OCI Patching in our Week 5 Blog (here) , lets now talk about automating Prechecks / Post checks , not all of them can be automated but significant portion of it can be done if attention is paid to Multiple use cases of Prechecks / Post checks

In a surgery, a doctor doesn’t walk into the OT and “figure it out.” Vitals are checked. Instruments are counted. Anesthesia readiness is confirmed. Post-procedure monitoring is predefined.

That level of discipline is expected in medicine. It should be expected in database operations too.

Prechecks and postchecks are are what separate controlled change from accidental impact. Not everything can be automated —
but a significant portion can, if designed intentionally.

1. Ground Zero: Where Challenges Start


+--------------------------------------------------------------------------------------------------+
| 1. Ground Zero: Where Challenges Start                                                          |
+--------------------------------------------------------------------------------------------------+
| - Prechecks live in emails, Word documents, or tribal knowledge                                 |
| - Postchecks depend on individual DBA experience                                                |
| - No standardized validation across environments                                                |
| - Evidence not captured consistently for audits                                                 |
| - Change windows consumed by manual verification                                                |
|                                                                                                 |
| Common Use Cases for Prechecks / Postchecks                                                     |
|                                                                                                 |
| • Patching Activities                                                                           |
|   - RMAN backup validation                                                                      |
|   - Invalid object count comparison                                                             |
|   - Tablespace & diskgroup free space                                                           |
|   - Cluster / CRS health                                                                        |
|   - AWR snapshot before & after                                                                 |
|                                                                                                 |
| • Deployment Activities                                                                         |
|   - Schema version validation                                                                   |
|   - Dependency verification                                                                     |
|   - Rollback script availability                                                                |
|   - Session spike monitoring post deployment                                                    |
|                                                                                                 |
| • Maintenance Activities                                                                        |
|   - DR sync status                                                                              |
|   - Archive log growth monitoring                                                               |
|   - ASM rebalance status                                                                        |
|   - Blocking session check                                                                      |
|                                                                                                 |
| • Upgrade / Migration Activities                                                                |
|   - COMPATIBLE parameter review                                                                 |
|   - Deprecated parameter scan                                                                   |
|   - Feature usage analysis                                                                      |
|                                                                                                 |
| • Security & Compliance Pre/Post Checks                                                         |
|   - Account lockout & password complexity profile validation                                    |
|   - Excessive privilege grants (DBA/SYS role review)                                            |
|   - Sensitive table / column access audit review                                                |
|   - Oracle Unified Auditing enabled & policy validation                                         |
|   - Audit retention policy enforcement                                                          |
|   - TDE key rotation status verification                                                        |
|   - Database Vault realm & rule validation                                                      |
|   - Fine-Grained Auditing (FGA) policy verification                                             |
|                                                                                                 |
| • Organizational / Process Validation                                                           |
|   - Change approval confirmation                                                                |
|   - Business window validation                                                                  |
|   - Stakeholder communication confirmation                                                      |
|                                                                                                 |
| >> Managed services often execute changes.                                                      |
|    Mature services validate readiness before and after execution.                               |
+-------------------------------------------------------------------------------------------------+

2. Underneath Ground Zero:


+--------------------------------------------------------------------------------------------------+
| 2. Underneath Ground Zero: Finding the Real Problem                                              |
+--------------------------------------------------------------------------------------------------+
| - Knowledge lives in people, not systems                                                         |
| - Checklists are not version-controlled                                                          |
| - No measurable readiness score                                                                  |
| - Validation steps skipped during “urgent” changes                                               |
| - DIY scripts automate execution — but not validation                                            |  
|                                                                                                  |
| Hidden Risks                                                                                     |
| - Patch applied on unstable baseline                                                             |
| - Deployment approved without rollback validation                                                |
| - DR drift unnoticed                                                                             |
| - Security misconfigurations discovered during audits                                            |
| - RCA becomes subjective (“we thought it was fine”)                                              |
|                                                                                                  |
| Core Issue                                                                                       |
| Execution gets automated.                                                                        |
| Validation does not.                                                                             |
|                                                                                                  |
| And without structured validation, predictability becomes fragile.                               |
+--------------------------------------------------------------------------------------------------+

3. Working Upwards:


+--------------------------------------------------------------------------------------------------+
| 3. Working Upwards: From Understanding to Solution                                               |
+--------------------------------------------------------------------------------------------------+
| - Convert recurring manual checks into structured automation modules                             |
| - Standardize precheck/postcheck templates across environments                                   |
| - Integrate validation into patching, deployment, and maintenance workflows                      |
| - Capture before/after state snapshots automatically                                             |
| - Produce readiness scorecards for every change                                                  |
| - Maintain auditable evidence logs                                                               |
| - Keep expert judgment manual — automate repeatable validation                                   |
|                                                                                                  |
| Managed Services Advantage                                                                       |
| • Institutional memory embedded into process                                                     |
| • Consistent validation across teams                                                             |
| • Faster, safer execution windows                                                                |
| • Audit-ready change documentation                                                               |
|                                                                                                  |
| DIY Limitation                                                                                   |
| • Automation focuses on “doing”                                                                  |
| • Mature services focus on “verifying”                                                           |
|                                                                                                  |
| >> In surgery, the checklist prevents avoidable mistakes.                                        |
|    In database operations, automated prechecks and postchecks prevent avoidable outages.         |
+--------------------------------------------------------------------------------------------------+

How Nabhaas helps you

If you’ve made it this far, you already sense there’s a better way — in fact, you have a way ahead.

TAB - Whitepaper ,
download here

Managed Delivery Services - Whitepaper ,
download here

Series Week 15/52 - OCI ExaCS, DB System, On Prem — Managed Service Complexity Handled

Abhilash Kumar | Oracle ACE ♠ — Sun, 08 Feb 2026 09:11:24 +0000

{ Abhilash Kumar Bhattaram : Follow on LinkedIn }

Treating Oracle Cloud Infrastructure (OCI) simply as an off-site extension of a local data center is the primary precursor to operational failure. In many migrations, databases are "lifted and shifted" into cloud services without a fundamental grasp of how those services operate internally. Teams often treat OCI DB Systems as standard on-premises VMs and view Exadata Cloud Service (ExaCS) merely as a larger, more powerful server. In these scenarios, redundancy, scalability, and resource limits are assumed to be inherent rather than engineered.

While the platform may appear stable initially, business growth inevitably exposes architectural flaws. Load patterns shift, session spikes occur, and storage costs begin to rise linearly because capacity was provisioned in advance rather than designed for on-demand flexibility. These issues frequently manifest as "cascading operational failures"—feedback loops where a small spike in errors or latency triggers a reduction in capacity, making the original problem worse and eventually requiring manual human intervention to recover.

Ultimately, what is often mislabeled as a "cloud technology failure" is actually a consequence of architectural ignorance. This article explores the recurring pitfalls encountered when cloud internals are ignored, demonstrating that a deep understanding of OCI database architectures is the only way to build predictable platforms and end the cycle of constant firefighting.

1. Ground Zero: Where Challenges Start

+--------------------------------------------------------------------------------------+
| Ground Zero: Where Challenges Start                                                  |
|--------------------------------------------------------------------------------------|
| - OCI DB Systems sized like on-prem VMs                                              |
| - ExaCS deployed but used as “big VM + storage”                                      |
| - Same architecture copied across Prod, UAT, DR                                      |
| - SLAs defined without knowing service-level constraints                             |
|                                                                                      |
| Repeated Pitfalls Seen in the Field:                                                 |
| • DB System chosen for high-concurrency apps → SESSION/PROCESS limits hit            |
| • ExaCS used but ASM redundancy not understood (HIGH vs EXTERNAL)                    |
| • Storage over-provisioned due to double redundancy assumptions                      |
| • Connection storms during sales / month-end not modeled                             |
| • Patching timelines copied without understanding rolling vs non-rolling             |
|                                                                                      |
| >> These don’t fail on Day 1 — they fail on Day 180                                  |
+--------------------------------------------------------------------------------------+

2. Underneath Ground Zero:

+--------------------------------------------------------------------------------------+
| Underneath Ground Zero: Finding the Real Problem                                     |
|--------------------------------------------------------------------------------------|
| - Architecture decisions made before workload is understood                          |
| - OCI services treated as infrastructure, not platforms                              |
| - Application compatibility ignored during DB version choices                        |
| - Business growth assumed to be linear                                               |
|                                                                                      |
| Patterns You’ve Written About Earlier:                                               |
| • SLAs defined without MTTR realism (Week 4)                                         |
| • Patch behavior differs across environments (Week 10)                               |
| • Non-prod testing doesn’t reflect prod load (Week 11)                               |
| • Databases multiplied instead of consolidated (Week 8)                              |
| • Global support teams unaware of local workload rhythm (Week 6)                     |
|                                                                                      |
| Root Cause:                                                                          |
| • Teams know Oracle DB — but not OCI DB services                                     |
| • Cloud internals discovered only during incidents                                   |
|                                                                                      |
| >> Most “cloud problems” are actually design problems                                |
+--------------------------------------------------------------------------------------+

3. Working Upwards:


+--------------------------------------------------------------------------------------+
| Working Upwards: From Understanding to Solution                                      |
|--------------------------------------------------------------------------------------|
| - Start from business rhythm (sales, month-end, reporting)                           |
| - Translate rhythm into workload patterns                                            |
| - Choose DB System vs ExaCS intentionally                                            |
| - Align ASM redundancy with service architecture                                     |
| - Design for peak concurrency, not average load                                      |
| - Validate assumptions using historical AWR, not guesses                             |
|                                                                                      |
| Practical Design Shifts:                                                             |
| • High concurrency + elastic growth → ExaCS                                          |
| • Predictable, steady workloads → DB System                                          |
| • Multiple apps → consolidate with isolation strategy                                |
| • Compliance-driven systems → architecture-first design                              |
|                                                                                      |
| CTO Outcome:                                                                         |
| • No redesign during scale                                                           |
| • Predictable RTO/RPO                                                                |
| • Controlled cloud costs                                                             |
| • Fewer “why is this happening now?” moments                                         |
|                                                                                      |
| >> Cloud predictability starts at design — not at incident response                  |
+--------------------------------------------------------------------------------------+

How Nabhaas helps you

If you’ve made it this far, you already sense there’s a better way — in fact, you have a way ahead.

TAB - Whitepaper ,
download here

Managed Delivery Services - Whitepaper ,
download here

Series Week 14/52 - Multi-Version, Multi-Environment: Where Services Matter Most

Abhilash Kumar | Oracle ACE ♠ — Mon, 02 Feb 2026 12:35:22 +0000

{ Abhilash Kumar Bhattaram : Follow on LinkedIn }

In large Oracle estates, applications rarely fail because of infrastructure alone. They fail when application behavior changes across database versions, patch levels,
and environments — silently breaking predictability at scale.

If one looks at Oracle EBS as an application stack , Oracle diligently provides certification matrix for each of its components , I have not seen this level of transparency in other ERP providers , so it's essentially the job of ERP provider and support team to make this kind of a matrix for implementations at client site.

A typical Oracle EBS compatability certification is shown here.

Now the problem comes when such details are not available and each stack of the application are treated as silos.

1. Ground Zero: Where Challenges Start

+--------------------------------------------------------------------------------------+
| 1. Ground Zero: Where Challenges Start                                               |
|--------------------------------------------------------------------------------------|
| - Application works in Dev/UAT but degrades or fails in Prod                         |
| - SQL behaves differently across DB versions (19c vs 23c)                            |
| - JDBC / OCI driver differences not aligned with DB patch levels                     |
| - Optimizer plan changes surface only during peak business hours                     |
| - Features tested in Non-Prod not supported or enabled in Prod                       |
| - Application teams test against availability, not production reality                |
|                                                                                      |
| >> Compatibility issues appear as performance bugs — not version mismatches.         |
+--------------------------------------------------------------------------------------+

2. Underneath Ground Zero:

+--------------------------------------------------------------------------------------+
| 2. Underneath Ground Zero: Finding the Real Problem                                  |
|--------------------------------------------------------------------------------------|
| - Optimizer behavior differs across RUs and major versions                           |
| - SQL semantics change due to deprecated or enhanced features                        |
| - Hard-coded assumptions (NLS, cursor sharing, adaptive plans) break silently        |
| - Patch-level drift across environments causes non-reproducible issues               |
| - Missing SQL Plan Baselines allow uncontrolled plan evolution                       |
| - No defined application-to-database compatibility matrix                            |
|                                                                                      |
| >> The issue isn’t instability — it’s unmanaged compatibility debt.                  |
+--------------------------------------------------------------------------------------+

3. Working Upwards:

+--------------------------------------------------------------------------------------+
| 3. Working Upwards: From Understanding to Solution                                   |
|--------------------------------------------------------------------------------------|
| - Define supported DB versions and RU levels per application                         |
| - Align Dev, UAT, and Prod on the same patching and upgrade cadence                  |
| - Use SQL Plan Baselines to preserve known-good execution plans                      |
| - Apply Real Application Testing to replay real production workloads                 |
| - Validate changes using AWR diff reports before promotion                           |
| - Embed compatibility checks into change and release management                      |
| - Treat services, not environments, as the unit of predictability                    |
|                                                                                      |
| >> Compatibility engineered upfront creates predictability at scale.                 |
+--------------------------------------------------------------------------------------+

How Nabhaas helps you

If you’ve made it this far, you already sense there’s a better way — in fact, you have a way ahead.

TAB - Whitepaper ,
download here

Managed Delivery Services - Whitepaper ,
download here

Oracle 19.28 - New features

Abhilash Kumar | Oracle ACE ♠ — Sun, 11 Jan 2026 16:54:23 +0000

{ Abhilash Kumar Bhattaram : Follow on LinkedIn }

Oracle 19.28 New Features

From 19.24 to 19.27 have been pretty silent on its new features , however in 19.28 its has introduced many new features.

The official documentation for the new features is here.

1. DBMS_DEVELOPER PL/SQL Packages : Provides a new PL/SQL package to retrieve object metadata as JSON (instead of XML), improving integration and performance for developers.

2. Enhancements to RADIUS Configuration : Updated RADIUS support with RFC 6613/6614 guidelines and TLS by default, improving MFA security.

3. IF NOT EXISTS Syntax Support : Adds IF EXISTS / IF NOT EXISTS syntax for DDL statements, reducing errors in scripts.

4. Multi-Factor Authentication (MFA) for Database Authentication : Native MFA support including push notifications and integration with MFA solutions (e.g., Cisco Duo, Oracle Mobile Authenticator) and PKI combos.

5. Oracle Database Cloud Backup Module for Azure Blob Storage : Enables RMAN backups directly to Azure Blob Storage (on-premises or Azure VM).

6. Oracle Update Advisor Support with Oracle Fleet Patching and Provisioning (FPP) : Integrates Oracle Update Advisor with FPP for automated patch recommendations and gold image management.

7. Schema Annotations : Allows storing/retrieving free-form metadata (name–value pairs) on schema objects — useful for application metadata.

8. SQL Diagnostic Report (NEW REPORT_SQL in DBMS_SQLDIAG) : New function that generates a detailed HTML diagnostic report for a specified SQL statement, including plan history, optimizer stats, index details, SQL Monitor info, etc.

Examples

I had a crack at 4 of the 8 new features that was released in 19.28
some were backported from 26ai

19.28 Feature - IF NOT EXISTS

This is a developer feature where objects can be created only oif not exists without errors

orcldev01> CREATE TABLE IF NOT EXISTS NABHAAS.TAB1 (sno number , name varchar2(40)) ;

Table created. 

Elapsed: 00:00:00.01
orcldev01> select owner,object_name,created from dba_objects where object_name like 'TAB1';

OWNER             |OBJECT_NAME                                       |CREATED
------------------|--------------------------------------------------|-----------------
NABHAAS           |TAB1                                              |11-JAN-2026 14:11

Elapsed: 00:00:00.00
orcldev01> CREATE TABLE IF NOT EXISTS NABHAAS.TAB1 (sno number , name varchar2(40)) ;

Table created. --> This is basically a dummy as table is already available

Elapsed: 00:00:00.00
orcldev01> select owner,object_name,created from dba_objects where object_name like 'TAB1';

OWNER             |OBJECT_NAME                                       |CREATED
------------------|--------------------------------------------------|-----------------
NABHAAS           |TAB1                                              |11-JAN-2026 14:11 --> You can see timestamp remains the same

Elapsed: 00:00:00.00
orcldev01>

19.28 Feature - dbms_developer

This is a 26ai backported feature where developers can view table data in JSON format

orcldev01> select dbms_developer.get_metadata (schema => 'NABHAAS' , name => 'TAB1' , object_type => 'TABLE' , level => 'BASIC') as metadata from dual;

METADATA
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
{"objectType":"TABLE","objectInfo":{"name":"TAB1","schema":"NABHAAS","columns":[{"name":"SNO","notNull":false,"dataType":{"type":"NUMBER"}},{"name":"NAME","notNull":false,"dataType":{"type":"VARCH
AR2","length":40,"sizeUnits":"BYTE"}}]},"etag":"DFB2EBA11BD001361E2DF1CCFE5C1BC9"}


orcldev01> select dbms_developer.get_metadata (schema => 'NABHAAS' , name => 'TAB1' , object_type => 'TABLE' , level => 'ALL') as metadata from dual;

METADATA
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
{"objectType":"TABLE","objectInfo":{"name":"TAB1","schema":"NABHAAS","columns":[{"name":"SNO","notNull":false,"dataType":{"type":"NUMBER"},"isPk":false,"isUk":false,"isFk":false,"hiddenColumn":fal
se,"virtualColumn":false,"identityColumn":false,"encryptedColumn":false},{"name":"NAME","notNull":false,"dataType":{"type":"VARCHAR2","length":40,"sizeUnits":"BYTE"},"isPk":false,"isUk":false,"isF
k":false,"hiddenColumn":false,"virtualColumn":false,"identityColumn":false,"encryptedColumn":false}],"hasBeenAnalyzed":false,"external":"NO","temporary":"NO","segmentCreated":"NO","dependencies":"
DISABLED","inMemory":"DISABLED","compression":"DISABLED"},"etag":"0AA002568416C9B3384480854A8160CC"}


Elapsed: 00:00:00.01
orcldev01> 


-- displaying in JSONM pretty format 
orcldev01> select 
json_serialize
( 
dbms_developer.get_metadata (schema => 'NABHAAS' , name => 'TAB1' , object_type => 'TABLE' , level => 'ALL')  pretty 
) as metadata from dual;  


METADATA
--------------------------------------------------------
{
  "objectType" : "TABLE",
  "objectInfo" :
  {
    "name" : "TAB1",
    "schema" : "NABHAAS",
    "columns" :
    [
      {
        "name" : "SNO",
        "notNull" : false,
        "dataType" :
        {
          "type" : "NUMBER"
        },
        "isPk" : false,
        "isUk" : false,
        "isFk" : false,
        "hiddenColumn" : false,
        "virtualColumn" : false,
        "identityColumn" : false,
        "encryptedColumn" : false
      },
      {
        "name" : "NAME",
        "notNull" : false,
        "dataType" :
        {
          "type" : "VARCHAR2",
          "length" : 40,
          "sizeUnits" : "BYTE"
        },
        "isPk" : false,
        "isUk" : false,
        "isFk" : false,
        "hiddenColumn" : false,
        "virtualColumn" : false,
        "identityColumn" : false,
        "encryptedColumn" : false
      }
    ],
    "hasBeenAnalyzed" : false,
    "external" : "NO",
    "temporary" : "NO",
    "segmentCreated" : "NO",
    "dependencies" : "DISABLED",
    "inMemory" : "DISABLED",
    "compression" : "DISABLED"
  },
  "etag" : "0AA002568416C9B3384480854A8160CC"
}


Elapsed: 00:00:00.01
orcldev01>

19.28 Feature - SQL Diag Report

This is my favourite , SQL Diag report provides an easy way to view data ( current / and historical ) in a single report. A zip file is created the the directory where the Diag report is run.

orcldev01> 

declare my_report clob;

begin
   my_report := dbms_sqldiag.report_sql('fa6yqnkq8fbm2', directory=>'NABHAAS_EXP', level=>'ALL');
end;
/


PL/SQL procedure successfully completed.

An example of how the report looks like.

19.28 Feature - Annotations

Annotations is similar to comments , though comments are still available annotations could be used going forward. Annotataions will be used by OML ( Oracle Machine Learning ) for look up values.

orcldev01> alter table NABHAAS.TAB1 annotations ( add audit_required 'YES');

Table altered.

Elapsed: 00:00:00.03
orcldev01> 

orcldev01> select object_name, object_type, annotation_name, annotation_value  from dba_annotations_usage;

OBJECT_NAME         |OBJECT_TYPE         |ANNOTATION_NAME                         |ANNOTATION_VALUE
--------------------|--------------------|----------------------------------------|--------------------
TAB1                |TABLE               |AUDIT_REQUIRED                          |YES

I feel of these 4 features are readily usable by DBA's and Developers.

How Nabhaas helps you

If you’ve made it this far, you already sense there’s a better way — in fact, you have a way ahead.

TAB - Whitepaper ,
download here

Managed Delivery Services - Whitepaper ,
download here

Series Week 13/52 - Scaling Oracle Managed Services Across Business Units

Abhilash Kumar | Oracle ACE ♠ — Sun, 11 Jan 2026 06:21:52 +0000

{ Abhilash Kumar Bhattaram : Follow on LinkedIn }

Closing the First 12 Weeks — From Stability to Scale
If you’ve been following the first 12 weeks of The Predictable CTO’s Journey from Nabhaas , you’ll notice a pattern.
Everything so far was about removing operational anxiety.

Patching that doesn’t steal weekends.
Managed delivery that works even when teams rotate.
Testing that actually behaves like production.
Compliance that’s handled quietly in the background, not during audits.

These are the fundamentals. Without them, nothing scales.

The next 12 weeks are different.
This is where Oracle stops being “one team’s problem” and starts becoming an enterprise service.

Multiple business units.
Different workloads.
Different peak cycles.
Different risk appetites.

As a founder, I’ve seen what breaks first when scale is attempted without discipline. This phase is about scaling Oracle operations without multiplying incidents, costs, or surprises.

Predictability is no longer about a single database.
It’s about giving you confidence across the organization.

Scaling Oracle Managed Services Across Business Units

As organizations grow, so do the expectations from Oracle services. What works for one team or application rarely scales to multiple lines of business. Silos form , complexity explodes, and reliability becomes localized rather than enterprise-level. This week we explore the operational challenges of scaling managed Oracle services and how to approach them methodically.

1. Ground Zero: Where Challenges Start


+--------------------------------------------------------------------------------------+
| - Each business unit runs its own Oracle instances with independent patch cycles     |
| - Teams use divergent standards for backup, DR, and deployment                       |
| - No unified inventory or service catalog across units                               |
| - Cross-business SLA expectations differ — no single “service level truth”           |
| - Incident response varies by team — inconsistent RCA quality                        |
| - Local optimizations hide global inefficiencies                                     |
| - Cost centers multiply without cost visibility                                      |
|                                                                                      |
| >> Scaling starts to break when more units mean more *isolated* processes, not more  |
|    *standardized* ones.                                                              |
+--------------------------------------------------------------------------------------+

2. Underneath Ground Zero:


+--------------------------------------------------------------------------------------+
| - Lack of centralized governance and policy — each unit builds its own               |
|   patch, backup, and change approach                                                 |
| - Duplication of roles and tools — infra, app, and DB teams solve the same problems  |
|   separately                                                                         |
| - Fragmented observability and metrics — no cross-unit SLA dashboard                 |
| - Multiple environments with inconsistent performance baselines                      |
| - Shadow IT provisioning bypasses central services                                   |
| - Legacy architecture boundaries restrict shared platforms                           |
| - No service catalog with clear ownership and billing tags                           |
|                                                                                      |
| >> The hidden problem is not just technical — it’s organizational: processes,        |
|    ownership, cost allocation, and shared visibility.                                |
+--------------------------------------------------------------------------------------+

3. Working Upwards:


+--------------------------------------------------------------------------------------+
| - Establish a single Oracle service catalog covering all business units              |
| - Standardize SLA definitions — uptime, MTTR, performance tiers                      |
| - Create a single inventory and governance model for patching, backups, and DR       |
| - Align change cycles across business units to reduce conflict and downtime          |
| - Centralize observability — aggregated AWR/ASH dashboards                           |
| - Define cross-unit incident response playbooks and RCA taxonomy                     |
| - Implement cost tagging (chargeback/showback) per unit                              |
| - Use managed automation (TAB) for repeatable patching, restores, and deployments    |
| - Quarterly service review with business unit stakeholders                           |
|                                                                                      |
| >> Managed services scale not by decentralizing — but by institutionalizing          |
|    repeatable processes that all units can trust.                                    |
+--------------------------------------------------------------------------------------+

How Nabhaas helps you

If you’ve made it this far, you already sense there’s a better way — in fact, you have a way ahead.

TAB - Whitepaper ,
download here

Managed Delivery Services - Whitepaper ,
download here

Series Week 12/52 - TAB fo OCI - Real life automation solutions

Abhilash Kumar | Oracle ACE ♠ — Sat, 20 Dec 2025 09:29:28 +0000

{ Abhilash Kumar Bhattaram : Follow on LinkedIn }

Lets take a look at our core Nabhaas offerring TAB ( Total Automation Box ) and some real life use cases

TAB for OCI Patching

I have blogged several articles about TAB ( whitepaper here ), here are some use cases that makes it one of the best tools for OCI automation

1. OCI Patching

Lets first understand the different types of OCI Patching for DB Systems and ExaCS class of OCI databases

1. Dom-0 Patching - ExaCS Infra patching 
2. Dom-U Patching - ExaCS VM patching 
3. GI - Patching - Grid Infra Patching  
4. DB - Patching - DB Realease Patching [Quarterly Patching]
5. DB - Upgrade - Database Upgrades

In all the above cases Dom-0 patching is OCI automated meaning scheduling is done at OCI console level and patching kicks off on its owm , there is really not much we can do here to monitor other than raising proactoive SR's and wait fo rpatch completions.

For the remaning 4 are Customer driven , meaning a customer has to decide when and how to do this and handle the risks assosciated with it.

Imagine a scenario like this in your non prods

devdb Current RU 19.21    Required RU 19.22
sitdb Current RU 19.21    Required RU 19.25
uatdb Current RU 19.21    Required RU 19.27

You have 3 different databases from 3 different compartments needed to be patched in 3 different versions due to application dependencies and all need to be done at the same time and you have only one weekend to do so.

TAB ( Total Automation Box ) solves exactly this problem by preparing simple run books to patching.

2. OCI Backup / Restores

Typical DBA activities are usually driven by manually prepared scripts with restores in On Prem databases , in OCI one can take a cloud backup and restore it as needed.

TAB helps prepare run books for backup restores so that weekend backups and restores are pre automated.

3. Database Deployments

Database deployments are semi automated taks whether they are on prod or on OCI as each application have thier dependencies , however TAB would help provide database deployments across databases if customized to each environment.

TAB gives you dozens of smart, automated ways to take care of your database needs—fast, accurate, and consistent.

How Nabhaas helps you

At Nabhaas, we work closely with teams to uncover dependencies, knowledge gaps, and process inefficiencies to ensure the patching cycle is smooth and predictable.

TAB ( Total Automation Box ) is how we automate patching lifecycles. https://www.nabhaas.com/tab

There is no staright answer to the points mentioned above but all of them needs to be addressed as best fits the organization.
At Nabhaas we ensure we identify all the above before beginning a patch cycle. Feel free to download our whitepaper here

Series Week 11/52 - Predictable Testing: Reducing Surprises in Prod with realistic Non Prod testing

Abhilash Kumar | Oracle ACE ♠ — Sun, 07 Dec 2025 06:33:45 +0000

{ Abhilash Kumar Bhattaram : Follow on LinkedIn }

Last week's blog was about "change management" , a natural next step is to look at how "production management" and how handling issues on the fly seems to be become accepted norm

In Oracle Production surprises rarely come from “new” issues — they come from changes that were never tested in conditions close to real business reality. Predictable testing is not about perfect test cases; it’s about testing the right workloads, at the

right scale, with visibility into how Oracle actually behaves.

In most Oracle environments, advanced capabilities like SQL Plan Baselines and Real Application Testing are rarely used to validate application changes under realistic conditions. Yet, these are some of the most powerful utilities Oracle provides — designed specifically to recreate production behavior in Non-Prod environments and allow teams to test changes with confidence, creativity, and control before they ever reach production.

1. Ground Zero:

+--------------------------------------------------------------------------------------+
| 1. Ground Zero: Where Challenges Start                                               |
|--------------------------------------------------------------------------------------|
| - Non-Prod environments too small to simulate real production load                   |
| - Real Application Testing (RAT) rarely used beyond demos                            |
| - SQL plan changes after patching go unnoticed                                       |
| - AWR snapshots taken, but never compared meaningfully                               |
| - Critical business SQL tested functionally, not performance-wise                    |
| - Month-end / quarter-end workloads never rehearsed                                  |
| - Test windows rushed, driven by change deadlines                                    |
|                                                                                      |
| >> Testing exists, but it does not resemble production reality.                      |
+--------------------------------------------------------------------------------------+

2. Underneath Ground Zero:

+--------------------------------------------------------------------------------------+
| 2. Underneath Ground Zero: Finding the Real Problem                                  |
|--------------------------------------------------------------------------------------|
| - Test data lacks production skew (hot blocks, data growth patterns)                 |
| - SQL Plan instability remains hidden until Prod load triggers it                    |
| - No SQL Plan Baselines to lock known-good execution plans                           |
| - AWR reports treated as static artifacts, not analytical tools                      |
| - RAT capture/replay skipped due to time or skill gaps                               |
| - Performance regressions discovered *after* users report slowness                   |
|                                                                                      |
| >> The problem isn’t lack of tools — it’s lack of workload-aware testing.            |
+--------------------------------------------------------------------------------------+

3. Working Upwards:

+--------------------------------------------------------------------------------------+
| 3. Working Upwards: From Understanding to Solution                                   |
|--------------------------------------------------------------------------------------|
| - Use Real Application Testing to capture real Prod workload patterns                |
| - Replay workloads on patched / upgraded environments before go-live                 |
| - Establish SQL Plan Baselines for business-critical SQL                             |
| - Run AWR diff reports to compare pre-change and post-change behavior                |
| - Test peak workloads (month-end, quarter-end, sales campaigns) explicitly           |
| - Validate I/O, CPU, latch, and wait events — not just query outputs                 |
| - Promote changes only after performance deltas are understood                       |
|                                                                                      |
| >> Predictable production starts with production-like testing.                       |
+--------------------------------------------------------------------------------------+

Used correctly, utilities like Real Application Testing and SQL Plan Baselines transform Non-Prod from a symbolic test bed into a production-like environment — where real workloads, real data patterns, and real performance behavior can be tested before business is impacted.

How Nabhaas helps you

At Nabhaas, we work closely with teams to uncover dependencies, knowledge gaps, and process inefficiencies to ensure the patching cycle is smooth and predictable.

TAB ( Total Automation Box ) is how we automate patching lifecycles. https://www.nabhaas.com/tab

There is no staright answer to the points mentioned above but all of them needs to be addressed as best fits the organization.
At Nabhaas we ensure we identify all the above before beginning a patch cycle. Feel free to download our whitepaper here

Series Week 10/52 — Integrated Change Management in Oracle Services

Abhilash Kumar | Oracle ACE ♠ — Sat, 29 Nov 2025 11:25:11 +0000

{ Abhilash Kumar Bhattaram : Follow on LinkedIn }

Regulatory compliance in Oracle environments is not a paperwork exercise — it is operational discipline. I had blogged about Week 9 Blog here that both Banks and Insurance companies RBI and IRDAI explicitly demand tight control over database security, audit trails, patching hygiene, DR readiness, and privileged access governance.

Most compliance failures happen NOT because teams lack tools — but because Oracle operations, business peaks, and regulatory clauses are not mapped to each other.

Some situations of bad change management in Oracle Databases.



 +--------------------------------------------------------------------------------------+
| Examples of Bad Change Management in Oracle Environments                             |
+--------------------------------------------------------------------------------------+
| 1. Uncoordinated Patching Cycles                                                     |
|--------------------------------------------------------------------------------------|
| - DEV on RU 19.29, UAT on RU 19.28 , PROD on 19.17 with missing PSUs                 |
| - SQL plans behave differently across environments                                   |
| - Bugs appear only in PROD because lower tiers are not aligned                       |
| >> Result: No predictable testing, no predictable performance                        |
+--------------------------------------------------------------------------------------+
| 2. Parameter Changes Without Governance                                              |
|--------------------------------------------------------------------------------------|
| - Sessions/processes updated by DBA during peak load without CAB approval            |
| - Requires restart → but business denies downtime                                    |
| - Leads to half-implemented changes, inconsistent runtime behavior                   |
| >> Result: Configuration drift and unexpected outages                                |
+--------------------------------------------------------------------------------------+
| 3. Schema Deployments Done Directly in Production                                    |
|--------------------------------------------------------------------------------------|
| - Developers run DDL directly on PROD to "fix an issue fast"                         |
| - Invalid objects, wrong grants, missing synonyms                                    |
| - No rollback scripts, no deployment evidence                                        |
| >> Result: PROD instability and audit non-compliance                                 |
+--------------------------------------------------------------------------------------+
| 4. Unreviewed Performance Fixes During Incidents                                     |
|--------------------------------------------------------------------------------------|
| - AWR-based patching of SQL hints directly in PROD                                   |
| - Lack of regression testing for critical business workloads                         |
| - Fixes become permanent without validation                                          |
| >> Result: Later month-end/quarter-end failures due to untested execution plans      |
+--------------------------------------------------------------------------------------+
| 5. DR & HA Configurations Drifting Out of Sync                                       |
|--------------------------------------------------------------------------------------|
| - Data Guard not patched when primary is patched                                     |
| - Lag increases, redo transport errors ignored                                       |
| - Switchover tests fail when actually needed                                         |
| >> Result: RTO/RPO commitments fail during real events                               |
+--------------------------------------------------------------------------------------+
| 6. No Unified Change Calendar Across Infra – App – DB                                |
|--------------------------------------------------------------------------------------|
| - App team deploys a new feature                                                     |
| - Middleware upgraded on a different weekend                                         |
| - DB parameters changed by DBA on another day                                        |
| >> Result: Outages caused by independent, conflicting changes                        |
+--------------------------------------------------------------------------------------+
| 7. Ad-Hoc Storage or ASM Rebalancing During Peak Hours                               |
|--------------------------------------------------------------------------------------|
| - Storage team expands LUNs without notifying DB team                                |
| - ASM starts auto-rebalance during business peak                                     |
| - Spikes I/O latency → application slowdown                                          |
| >> Result: “Database is slow” escalations with no clear RCA                          |
+--------------------------------------------------------------------------------------+
| 8. No Version Control for Database Objects                                           |
|--------------------------------------------------------------------------------------|
| - Different object definitions between DEV / UAT / PROD                              |
| - Invalid or outdated packages deployed unknowingly                                  |
| - No tracking of who changed what and when                                           |
| >> Result: Debugging failures becomes impossible                                     |
+--------------------------------------------------------------------------------------+
| >> Poor change management doesn’t cause incidents — it *accumulates* them.           |
+--------------------------------------------------------------------------------------+

1. Ground Zero: Uncontrolled Change Management

Let us understand the uncontrolled chaos

+--------------------------------------------------------------------------------------+
| 1. Ground Zero: Where Change Chaos Begins                                            |
+--------------------------------------------------------------------------------------+
| - DB, App, Infra teams follow separate change cycles — no shared calendar            |
| - Changes about to go live without dependency mapping                                |
| - Emergency fixes bypass standard review & approvals                                 |
| - No unified deployment history or audit trail across stack                          |
| - Pre-checks or impact analysis often skipped, especially under time pressure        |
| - Rollback plans missing or untested — causing extended downtime on failures         |
+--------------------------------------------------------------------------------------+

2. Underneath Ground Zero: The root cause of change drift

There needs to be change manegement system that is relevant to the organization , using ready products to plug play for database changes is not the right way. The change management should be in line with techinical teams who are able to follow them.

+--------------------------------------------------------------------------------------+
| 2. Underneath Ground Zero: Understanding the Hidden Risk                             |
+--------------------------------------------------------------------------------------+
| - Risk not visible until after change — no predictive risk rating                    |
| - No correlation between change events and past incidents                            |
| - Multiple independent change tools & ticketing systems — no traceability            |
| - Environments (Dev/QA/Prod) inconsistent — tests pass in lower env, fail in Prod    |
| - Lack of schema / DB / infra / network dependency maps                              |
| - No baseline comparison (performance, capacity, configs) before/after changes       |
|                                                                                      |
| >> The problem isn’t changes themselves — it's unmanaged change process that         |
|    turns changes into outages.                                                       |
+--------------------------------------------------------------------------------------+

3. Working Upwards: Building a Managed Change-First Oracle Operation

To identify a change process one needs to understand how the bussiness works and ot do that leadership at every layer should provide inputs for maintanbility of the systems.

+--------------------------------------------------------------------------------------+
| 3. Working Upwards: Building a Managed Change-First Oracle Operation                 |
+--------------------------------------------------------------------------------------+
| - Establish unified Change Calendar (DB, App, Infra, Cloud)                          |
| - Require Dependency Mapping & Impact Analysis before any change                     |
| - Implement Automated Pre-Change Checks (space, backups, code quality, configs)      |
| - Use Version-controlled Deployments + Rollback-ready packages                       |
| - Enforce Post-Change Validation (performance baseline, functionality smoke tests)   |
| - Maintain central Change + Audit History for traceability and compliance            |
| - Introduce “Change Risk Scoring” based on past impact & complexity                  |
| - Align change windows with business load calendar — avoid high-traffic periods      |
|                                                                                      |
| >> Change management should not be an afterthought — it must be the backbone         |
|    of stable, predictable Oracle operations.                                         |
+--------------------------------------------------------------------------------------+

How Nabhaas helps you

At Nabhaas, we work closely with teams to uncover dependencies, knowledge gaps, and process inefficiencies to ensure the patching cycle is smooth and predictable.

TAB ( Total Automation Box ) is how we automate patching lifecycles. https://www.nabhaas.com/tab

There is no staright answer to the points mentioned above but all of them needs to be addressed as best fits the organization.
At Nabhaas we ensure we identify all the above before beginning a patch cycle. Feel free to download our whitepaper here