DEV Community

Abdullah Qatan
Abdullah Qatan

Posted on

Al-Diwan Al-Shamil is open source

#javascript #android #pwa #opensource

Diwan Shamil — An Open Arabic Reading and Research Library
Diwan Shamil is an open-source Arabic reading and research project that brings together the Holy Quran, selected Hadith collections, Arabic poetry, and built-in Arabic search tools in one fast and focused application.
The project is available as a web application and Progressive Web App (PWA), with support for offline reading and the ability to build an Android APK using Capacitor.
Main Features

  • Read the Holy Quran with a Surah index and in-text search.
  • Search through multiple Arabic Hadith collections.
  • Browse more than 75,000 Arabic poems.
  • Progressive loading for large poetry collections.
  • Save Quran verses, Hadith, and poems to personal bookmarks.
  • Save and restore reading positions.
  • Offline reading and searching after downloading the library.
  • Four visual themes: Sand, Night, Green, and Paper.
  • Adjustable font size.
  • Full Arabic RTL interface.
  • Built-in Arabic word explanations and lexicon data.
  • Export and import bookmarks and reading positions.
  • Responsive design for phones, tablets, and desktop browsers.
  • No user account or private backend is required for the core features.
  • Local storage for bookmarks, reading positions, settings, and backups. Reusing the Project and Its Files Developers and project owners can use Diwan Shamil as a foundation for building:
  • Arabic reading and research libraries.
  • Offline Quran, Hadith, poetry, or cultural applications.
  • Installable Progressive Web Apps.
  • Android applications built from web technologies.
  • Arabic search and text-analysis tools.
  • Educational and cultural applications based on JSON data.
  • Customized versions with different branding, interfaces, or datasets. The source code, scripts, project structure, and selected original materials can be reused according to the project license. Data files must be reviewed separately because the repository contains third-party datasets with different licenses and attribution requirements. The entire repository is not covered by one universal license. It is important to distinguish between the original project code and third-party data. Licensing Project Code The original application code, scripts, and clearly identified project materials are licensed under the MIT License. The MIT License allows users to use, copy, modify, merge, publish, distribute, sublicense, and sell the code, provided that the copyright notice and license text are preserved. The main license file is: LICENSE Quran Data The local Quran data is based on the quran-uthmani-quran-academy edition provided through Al Quran Cloud. The source and usage information are documented in: data/QURAN-SOURCES.md data/quran-manifest.json The project also includes SHA-256 hashes and a reproducible import script for rebuilding the local Quran files. Hadith Data The Hadith collections are subject to their respective source terms. The repository documents the data under:
  • ODbL 1.0 for the database.
  • DbCL 1.0 for database contents. Relevant files include: data/HADITH-SOURCES.md data/QURANLAB-LICENSES.md THIRD-PARTY-NOTICES.md Attribution notices and source license requirements must be preserved when redistributing the data or derived versions. Poetry Data The poetry corpus is located in: data/poetry/ Related license and attribution files include: data/POETRY-LICENSE data/poetry/NOTICE.md data/poetry/POETRY-LICENSE-SHAWQI The project does not claim that every poem is public domain or freely reusable in every country. Anyone planning commercial redistribution should review the status of the individual texts and the source database. Lexicon Data The project includes multiple lexicon sources, including Arabic WordNet, Kaikki/Wiktionary Arabic data, and original project-written definitions and notes. Documentation is available in: data/lexicon/README.md data/lexicon/ORIGINAL-GLOSSES-LICENSE.md data/lexicon/coverage/NOTICE.md Different lexicon components may be subject to CC BY 4.0, CC BY-SA 4.0, or the project’s MIT license for original materials. Source attribution and ShareAlike requirements must be preserved where applicable. Complete Third-Party Notice Before redistributing the project, its data, or an APK in a commercial product, review: THIRD-PARTY-NOTICES.md This file explains the difference between the MIT license for the project code and the separate licenses applicable to third-party data. Technical Architecture The project uses lightweight web technologies without a large application framework:
  • HTML
  • CSS
  • JavaScript
  • JSON
  • Progressive Web App
  • Service Worker
  • Cache API
  • Capacitor
  • Android Gradle Main Files
  • index.html: Application entry point, SEO metadata, and Service Worker registration
  • app.js: Reading, search, bookmarks, reading positions, settings, and download logic
  • styles.css: Responsive interface, themes, and RTL styling
  • sw.js: Offline caching and Service Worker logic
  • manifest.json: PWA metadata, application name, and icons
  • data/: Quran, Hadith, poetry, lexicon, metadata, and license files
  • scripts/: Import, build, verification, and release scripts
  • android/: Capacitor Android project and Gradle configuration
  • .github/workflows/: Automated Android build and release workflow
  • LICENSE: MIT license for project code
  • THIRD-PARTY-NOTICES.md: Third-party data and license overview
  • PRIVACY.md: Local storage and privacy documentation Data Structure Quran data is stored in: data/quran.json data/surah-meta.json data/quran-manifest.json Hadith collections are stored in JSON files beginning with: data/ara- Poetry data is stored in: data/poetry/ The poetry collection is divided into multiple JSON parts to support progressive loading. Lexicon data is stored in: data/lexicon/ This directory includes core lexicon files, original project definitions, Wiktionary/Kaikki-derived data, coverage files, and source notices. Large data files should not be edited manually without updating their source documentation, metadata, and verification steps. Running the Website Locally The project should be served through a local HTTP server instead of opening index.html directly with file://. With Python installed, run: python3 -m http.server 4173 Then open: http://localhost:4173 A different static file server can also be used as long as it serves the project root and supports JSON files and Service Worker registration from the same origin. Project Commands Install dependencies: npm ci Run JavaScript and script syntax checks: npm run verify Run legal notice and data provenance checks: npm run verify:legal Verify lexicon coverage and derived data: npm run verify:lexicon Rebuild the Quran data from the documented source: npm run import:quran Prepare the web files for Android: npm run prepare:web Build a debug Android version: npm run android:build Android Build The Android application uses Capacitor to package the web application as a native Android application. A local Android build generally requires:
  • Node.js
  • JDK 21
  • Android SDK
  • Android build tools
  • Gradle support through the Android project The project includes helper scripts such as: scripts/build-release.sh scripts/sign-release-apk.sh scripts/prepare-web.sh Release signing credentials and keystore files must remain outside the repository and must be configured through a secure build environment. They should never be committed to Git or included in public documentation. Publishing and Releases The web application can be deployed to any static hosting provider that supports HTTPS. HTTPS is recommended for reliable Service Worker behavior outside local development. The repository includes a GitHub Actions workflow that can build and sign Android releases, verify the APK signature, upload build artifacts, and create a GitHub Release. Signing credentials are kept outside the repository in protected CI settings. The public project documentation does not contain private keys, passwords, keystore files, or API credentials. Privacy The application does not require a user account or private backend for its core features. Bookmarks, reading positions, settings, and local backups are stored on the user’s device. The current code does not upload backup files to a private server. The project does not include visible analytics or behavioral tracking in the reviewed code. The privacy documentation is available at: PRIVACY.md This document should be updated if authentication, cloud synchronization, analytics, or a new external service is added. Contributing Contributions are welcome in areas such as:
  • Arabic search improvements.
  • Interface and accessibility improvements.
  • Mobile performance.
  • Offline caching.
  • New documented data sources.
  • Lexicon and text-processing tools.
  • Android and PWA improvements.
  • Testing and verification scripts. When adding a new dataset, document:
  • The dataset name.
  • The original source.
  • The license.
  • Redistribution terms.
  • Any transformations or modifications.
  • The file containing the attribution notice. After making changes, run: npm run verify npm run verify:legal npm run verify:lexicon Important Limitations Diwan Shamil is a reading and research application. It does not provide religious rulings, independently authenticate Hadith, or replace specialized scholarly references and dictionaries. The project also does not claim that every poem, edition, typesetting choice, or derived dataset is reusable in every jurisdiction. Public availability on GitHub does not mean that every file in the repository is covered by the MIT License. Project Links
  • GitHub Repository: https://github.com/Abdullah-Qatan-AQ/Diwan-Shamil
  • Releases and APK Files: https://github.com/Abdullah-Qatan-AQ/Diwan-Shamil/releases
  • Full Arabic Project Guide: https://github.com/Abdullah-Qatan-AQ/Diwan-Shamil/blob/main/PROJECT-GUIDE-AR.md
  • MIT License: https://opensource.org/license/mit
  • Al Quran Cloud Terms: https://alquran.cloud/terms-and-conditions
  • Al Quran Cloud API Documentation: https://alquran.cloud/api
  • ODbL 1.0: https://opendatacommons.org/licenses/odbl/1-0/
  • DbCL 1.0: https://opendatacommons.org/licenses/dbcl/1-0/
  • CC BY 4.0: https://creativecommons.org/licenses/by/4.0/
  • CC BY-SA 4.0: https://creativecommons.org/licenses/by-sa/4.0/ Summary Diwan Shamil is an open and extensible Arabic reading project that combines reading, search, bookmarks, reading positions, lexicon support, and offline access in one application. The project can be used as a foundation for new Arabic libraries, cultural applications, educational tools, PWA projects, and Android applications, provided that the MIT license is preserved for the original code and that every third-party dataset is reused according to its own license and attribution requirements. Code: MIT Data: Multiple source-specific licenses Platforms: Web / PWA / Android Interface: Arabic RTL Offline Support: Available

Top comments (0)