The Engineering Math Behind Enthalpy Calculations: From Psychrometric Formulas to Coil Load Determination

Did you know that at 0°C with zero moisture content, specific enthalpy is defined as 0 kJ/kg by convention? This reference point means that cold, dry air can have negative enthalpy values—a physical reality that often surprises engineers working with HVAC systems in arctic climates or industrial refrigeration applications.

The Formula: Breaking Down the Enthalpy Calculation

Specific enthalpy (h) represents the total heat content of moist air per unit mass of dry air, combining sensible heat from temperature and latent heat from moisture. The fundamental formula in metric units is:

// Metric enthalpy calculation
psat_kPa = 0.61078 * Math.exp(17.625 * tdb / (243.04 + tdb));
P_v = (rh / 100) * psat_kPa;
W = 621.945 * P_v / (101.325 - P_v);
sensible = 1.006 * tdb;
latent = (W / 1000) * (2501 + 1.86 * tdb);
enthalpy = sensible + latent;

The saturation pressure term uses the Magnus approximation, which accurately models water vapor pressure across the -60°C to 65°C range. The 17.625 and 243.04 constants are empirically derived for water vapor. The humidity ratio (W) calculation converts relative humidity to actual moisture content using the ideal gas law for water vapor in air. The sensible component (1.006 * tdb) represents the specific heat of dry air at constant pressure, while the latent term includes both the heat of vaporization (2501 kJ/kg at 0°C) and the specific heat of water vapor (1.86 kJ/kg·K).

Why does the formula include temperature dependence in the latent term? Because the latent heat of vaporization decreases with increasing temperature—from 2501 kJ/kg at 0°C to about 2406 kJ/kg at 40°C. The (2501 + 1.86 * tdb) term captures this physical reality, making the calculation accurate across the full temperature range. The division by 1000 converts W from g/kg to kg/kg, ensuring proper unit consistency in the latent heat calculation.

Worked Example 1: Cooling Coil Load Calculation

Let's calculate the enthalpy difference for a cooling coil with entering air at 35°C dry-bulb and 60% relative humidity, leaving at 12°C dry-bulb and 90% relative humidity. First, we calculate entering conditions:

// Entering air (35°C, 60% RH)
psat1 = 0.61078 * Math.exp(17.625 * 35 / (243.04 + 35)); // 5.624 kPa
P_v1 = (60 / 100) * 5.624; // 3.374 kPa
W1 = 621.945 * 3.374 / (101.325 - 3.374); // 21.42 g/kg
sensible1 = 1.006 * 35; // 35.21 kJ/kg
latent1 = (21.42 / 1000) * (2501 + 1.86 * 35); // 54.89 kJ/kg
enthalpy1 = 35.21 + 54.89; // 90.10 kJ/kg

Now for leaving conditions:

// Leaving air (12°C, 90% RH)
psat2 = 0.61078 * Math.exp(17.625 * 12 / (243.04 + 12)); // 1.402 kPa
P_v2 = (90 / 100) * 1.402; // 1.262 kPa
W2 = 621.945 * 1.262 / (101.325 - 1.262); // 7.84 g/kg
sensible2 = 1.006 * 12; // 12.07 kJ/kg
latent2 = (7.84 / 1000) * (2501 + 1.86 * 12); // 19.77 kJ/kg
enthalpy2 = 12.07 + 19.77; // 31.84 kJ/kg

The enthalpy difference Δh = 90.10 - 31.84 = 58.26 kJ/kg. For an airflow of 2.5 m³/s, the total cooling load is:

airflow = 2.5; // m³/s
density = 1.2; // kg/m³
totalLoad = (airflow * density * 58.26) / 3600; // 48.55 kW

Worked Example 2: Heating with Humidification

Consider a winter heating scenario where outdoor air at -10°C and 80% RH is heated to 20°C without adding moisture. First, calculate outdoor enthalpy:

// Outdoor air (-10°C, 80% RH)
psat_out = 0.61078 * Math.exp(17.625 * -10 / (243.04 + -10)); // 0.260 kPa
P_v_out = (80 / 100) * 0.260; // 0.208 kPa
W_out = 621.945 * 0.208 / (101.325 - 0.208); // 1.28 g/kg
sensible_out = 1.006 * -10; // -10.06 kJ/kg
latent_out = (1.28 / 1000) * (2501 + 1.86 * -10); // 3.18 kJ/kg
enthalpy_out = -10.06 + 3.18; // -6.88 kJ/kg

After heating to 20°C (humidity ratio remains 1.28 g/kg):

// Heated air (20°C, same W)
sensible_heated = 1.006 * 20; // 20.12 kJ/kg
latent_heated = (1.28 / 1000) * (2501 + 1.86 * 20); // 3.25 kJ/kg
enthalpy_heated = 20.12 + 3.25; // 23.37 kJ/kg

The enthalpy increase Δh = 23.37 - (-6.88) = 30.25 kJ/kg. For 1000 CFM airflow, using the Imperial formula with proper unit conversion:

// Imperial calculation
W_imperial = 1.28 * 7; // 8.96 gr/lb (conversion factor: 7000 gr/lb ÷ 1000 g/kg)
enthalpy_imperial = 0.240 * 68 + (8.96 / 7000) * (1061 + 0.444 * 68); // 13.03 BTU/lb
airflow_cfm = 1000;
heatingLoad = 4.5 * airflow_cfm * 30.25 * 0.4299; // 58.5 kBtu/hr (0.4299 converts kJ/kg to BTU/lb)

What Engineers Often Miss

First, the 7,000 divisor in Imperial calculations is frequently overlooked. When humidity ratio is measured in grains per pound (gr/lb), it must be divided by 7,000 to convert to pounds of moisture per pound of dry air. Omitting this division makes the latent term 7,000 times too large—a common error that leads to dramatically incorrect coil sizing.

Second, engineers often confuse which airflow to use in mixed-air systems. For ventilation energy calculations, only the outdoor airflow should be used with the enthalpy difference between outdoor and supply conditions. Using total supply airflow overestimates energy transfer by including recirculated air that doesn't undergo the full enthalpy change.

Third, the temperature dependence of latent heat is frequently ignored in simplified calculations. While 2,501 kJ/kg is accurate at 0°C, it decreases to about 2,408 kJ/kg at 40°C. The (2501 + 1.86 * tdb) term in the formula accounts for this variation, ensuring accuracy across the full -60°C to 65°C range. Using a constant value introduces errors of up to 4% in high-temperature applications.

Try the Calculator

For accurate enthalpy calculations across different moisture input types and both metric and imperial units, use the dedicated Enthalpy Calculator. It handles all the unit conversions and formula variations automatically, reducing calculation errors and saving valuable engineering time.