<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Water System Guide</title>
    <description>The latest articles on DEV Community by Water System Guide (@watersystemguide).</description>
    <link>https://dev.to/watersystemguide</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4206668%2F25181ce7-e805-4099-9618-d4b3b62509ad.png</url>
      <title>DEV Community: Water System Guide</title>
      <link>https://dev.to/watersystemguide</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/watersystemguide"/>
    <language>en</language>
    <item>
      <title>How we built a source-checked dataset of 160 US hard water and water quality statistics</title>
      <dc:creator>Water System Guide</dc:creator>
      <pubDate>Sun, 25 Oct 2026 13:29:55 +0000</pubDate>
      <link>https://dev.to/watersystemguide/how-we-built-a-source-checked-dataset-of-160-us-hard-water-and-water-quality-statistics-273n</link>
      <guid>https://dev.to/watersystemguide/how-we-built-a-source-checked-dataset-of-160-us-hard-water-and-water-quality-statistics-273n</guid>
      <description>&lt;p&gt;Water System Guide (&lt;a href="https://watersystemguide.com/" rel="noopener noreferrer"&gt;https://watersystemguide.com/&lt;/a&gt;) publishes a page of US hard water and drinking water statistics. We also released the numbers as an open dataset: 160 rows, one statistic per row, CC BY 4.0 for the compilation (DOI &lt;a href="https://doi.org/10.5281/zenodo.23534609" rel="noopener noreferrer"&gt;10.5281/zenodo.23534609&lt;/a&gt;). This post covers how it was built and what it can't tell you.&lt;/p&gt;

&lt;h2&gt;
  
  
  The problem with water statistics
&lt;/h2&gt;

&lt;p&gt;Many water figures get repeated from blog to blog with no traceable origin. "85% of US homes have hard water" is the best-known one. We could not trace it to a primary source, so it is not in the dataset. The rule was simple: if a figure can't be found at the body that produced it, it stays out.&lt;/p&gt;

&lt;h2&gt;
  
  
  Sources
&lt;/h2&gt;

&lt;p&gt;Every row cites the organization that produced the figure: federal agencies (EPA, USGS, CDC), state agencies, utilities, universities, WHO/UNICEF, original survey publishers (Gallup, WQA) and the foundation that commissioned a study (WQRF/Battelle). No roundup blogs or vendor pages.&lt;/p&gt;

&lt;p&gt;The 160 rows come from 64 source URLs and 39 publishers. By source type:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;source_type&lt;/th&gt;
&lt;th&gt;rows&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;us_federal_agency&lt;/td&gt;
&lt;td&gt;91&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;us_local_government_or_utility&lt;/td&gt;
&lt;td&gt;25&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;us_state_agency&lt;/td&gt;
&lt;td&gt;15&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;industry_association_or_foundation&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;intergovernmental_organization&lt;/td&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;survey_publisher&lt;/td&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;university&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;They cover 12 topics, from chloride regulation (22 rows) and public water systems (20) to global access (8).&lt;/p&gt;

&lt;h2&gt;
  
  
  Verification
&lt;/h2&gt;

&lt;p&gt;On 2026-10-11 each of the 64 source URLs was fetched. HTML was converted to text and PDFs with &lt;code&gt;pdftotext -layout&lt;/code&gt;; spreadsheet values were read from the original file. Each statistic's exact quote or table value was then matched against the fetched text, with whitespace normalized. Anything that didn't match was rewritten to what the source says, or dropped. The final pass reported 0 mismatches, and every URL returned HTTP 200 at build time. One CDC page blocks automated requests, so it was read with a browser-style reader, and those rows say so.&lt;/p&gt;

&lt;h2&gt;
  
  
  Cleaning choices
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Dated, not blended.&lt;/strong&gt; Each row keeps its own source date. Revised series stay as separate rows: EPA's lead service line estimate appears as 9.2M (2023), 9M (2024) and 4M (2025).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;No machine-split numbers.&lt;/strong&gt; Each statistic is a full sentence. Splitting it into &lt;code&gt;value&lt;/code&gt; and &lt;code&gt;unit&lt;/code&gt; columns automatically wasn't reliable enough to publish, so the sentence is the authoritative value.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;License per row.&lt;/strong&gt; A &lt;code&gt;source_license_note&lt;/code&gt; column separates US federal figures (91 rows, public domain), state/local government figures (40 rows, facts cited with credit) and figures from copyrighted sources (29 rows, referenced with credit).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Caveats in the data.&lt;/strong&gt; 29 rows carry &lt;code&gt;review_flags&lt;/code&gt;, for example a source with no publication date, or an industry-affiliated source.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A quick look with the standard library:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;csv&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;collections&lt;/span&gt;
&lt;span class="n"&gt;rows&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;list&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;csv&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;DictReader&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;data.csv&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;encoding&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;utf-8&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)))&lt;/span&gt;
&lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;  &lt;span class="c1"&gt;# 160
&lt;/span&gt;&lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;collections&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Counter&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;topic&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;most_common&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Limits
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;It is a compilation of published figures, not new measurements or a statistical sample.&lt;/li&gt;
&lt;li&gt;Rows differ in year, method, unit and geography (102 rows are US-wide, 10 global, the rest state or local). Read &lt;code&gt;source_date&lt;/code&gt; and the source before comparing two rows.&lt;/li&gt;
&lt;li&gt;Source pages can change after the 2026-10-11 retrieval.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Where to get it
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Companion page: &lt;a href="https://watersystemguide.com/guides/hard-water-statistics/" rel="noopener noreferrer"&gt;https://watersystemguide.com/guides/hard-water-statistics/&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Dataset (Zenodo): &lt;a href="https://doi.org/10.5281/zenodo.23534609" rel="noopener noreferrer"&gt;https://doi.org/10.5281/zenodo.23534609&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Kaggle: &lt;a href="https://www.kaggle.com/datasets/watersystemguide/us-hard-water-and-water-quality-statistics-2026" rel="noopener noreferrer"&gt;https://www.kaggle.com/datasets/watersystemguide/us-hard-water-and-water-quality-statistics-2026&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you use a single figure, please cite the original publisher in that row too.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Disclosure: this post is published by Water System Guide. It was drafted with AI assistance and reviewed by the site owner. All figures come from the dataset files and its documentation.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>opendata</category>
      <category>python</category>
      <category>datacleaning</category>
      <category>dataset</category>
    </item>
  </channel>
</rss>
