DEV Community: ANYIGOR TOBIAS MMADUECHENDU

Debugging docker-compose errors

ANYIGOR TOBIAS MMADUECHENDU — Tue, 11 Jun 2024 12:58:07 +0000

Introduction:

Ever faced challenges while trying to run multi-containers with Docker?
I was trying to operationalize my model monitoring and evaluation using Evidently, and had to create a setup to run multi-containers using Docker Compose. I was not expecting a smooth run when I ran

docker-compose --build

in my terminal.

Errors covered are:

Top-level object must be a mapping
Additional property volume is not allowed
Volumes must be a mapping
* error decoding ‘ports’ : Invalid containerPort: 5432

Top-level object must be a mapping:

If you get this error, "the top-level object must be a mapping" , when you run docker-compose up–build, you need to check your top-level elements, such as name, services, volume, version, e.t.c. Specification of containers are made under applicable top-level elements.
In my case, my top-level elements were all intact. I got to the next bug when I closed the docker-compose.yaml file and executed the docker-compose –build command from the terminal.

Resources:
Stackoverflow: gave me the idea of closing the file and running again
Docker documentation: helped to understand what top level elements are
The Docker forum on the bug: was not useful to me.

Additional property “volumnes” is not allowed

docker-compose.yml: services_grafana Additional property volumnes is not allowed
You will get an error message, similar to the one above if you add an unacceptable term in your configuration. In my case, I got the error because I typed in “volumnes” instead of “volumes” under the grafana container set-up.

I had to read the error carefully to see the mistake.

Resolving this moved me to the next error.

Resources I consulted:
Stackoverflow: Reading the challenges of others helped me to understand what could be the cause of the error, although my challenge was not there.
Docker forum: this was not so helpful

Error: Volumes must be a mapping

.\docker-compose.yml:volumes must be a mapping

This error is due to the wrong indentation of the volume specification (as a top-level element) or a mapping (under a top-level element).

Implementing the right indentations will solve the problem.

Resource
Stackoverflow: The information here was enough

error decoding ‘ports’ : Invalid containerPort: 5432

* error decoding ‘ports’ : Invalid containerPort: 5432

The PostgreSQL database uses port 5432 by default. You must ensure you are using the right port. Inspect your port specifications and indentation, close your yaml file, and run docker-compose-build in your terminal.

docker-compose-build

Reading your codes and understanding the tool you’re working with will reduce the hours you are likely to spend trying to get things to work. Always read documentation to understand the features, and use public forums like stackoverflow.

The docker-compose.yml file

version: '3.7'

volumes: 
  grafana_data: {}

networks:
  front-tier:
  back-tier:

services:
  db:
    image:  postgres
    restart: always
    environment:
      POSTGRES_PASSWORD: Bender
    ports:
      - "5432:5432"
    networks:
      - back-tier

  adminer:
    image:  adminer
    restart: always
    ports:
     - "8080:8080"
    networks:
      - back-tier
      - front-tier

  grafana:
    image: grafana/grafana
    user: "472"
    ports:
      - "3000:3000"
    volumes:
      - ./config/grafana_datasources.yaml:/etc/grafana/provisioning/datasources/datasources.yaml:ro
    networks:
      - back-tier
      - front-tier
    restart: always

Build a YouTube Video Downloader Using PyTube And Streamlit

ANYIGOR TOBIAS MMADUECHENDU — Sun, 22 Oct 2023 00:46:13 +0000

I was inspired to build a simple application for downloading YouTube videos when I could no longer stand the endless ads. I could not get an alternative to downloading YouTube content without irritating ads. This tutorial will guide you on how to build exactly the same application in a simple step-by-step guide, all you need is a basic knowledge of Python, Streamlit, and Pytube.

Here is a summary of this tutorial:

Introduction
Step One: setting up a virtual environment
Step Two: how to install all required packages
Step Three: importing all required packages
Step Four: build a basic Streamlit app
Step Five: Download videos and audio from YouTube using PyTube
Conclusion
References

Introduction

Pytube is a Python package for downloading YouTube content. This package comes with classes and methods for downloading YouTube videos according to preferred resolutions, and also for filtering audio content. Streamlit makes it easier to build a front-end web application. We will build the YouTube Downloader App using both packages.

Step One: Setting up a virtual environment

A virtual environment is necessary for dependency management. A virtual environment makes it possible to install and work different versions of packages, all in one computer. You can decide to build a project with Python 3.8 and another with Python 3.10, with the help of a virtual environment. There are different packages for creating a virtual environment such as using:

conda -pipenv -venv -poetry Read this article to learn more about how to create a virtual environment using pipenv, activate, install and deactivate packages using pipenv. We will set up the virtual environment for this project with pipenv. Navigate to the preferred directory for your project and write the code below and press enter

pipenv --python 3.10

Here is a sample of the same in an anaconda terminal

Ensure you create a directory for the project before you create a virtual environment. Read this article if you do not know how to do that.

Step Two: how to install all required packages

This project requires only Streamlit, for the web app interface, streamlit-extras, and PyTube, a package for downloading and extracting YouTube content. Install them using the codes below

pipenv install streamlit streamlit-extras pytube

observe the space between the name of each package, do not write them together or use any other delimiter.

Avoid ModuleNotGoundError: No module named 'altair_vegalite.v4' by installing a version of altair less than the version 5

Install the required altair package using the code below
pipenv install "altair<5"
Next, we will go to a preferred code editor, such as Pycharm, VSCode, and create a Python script. All Python scripts must end with .py.

Step 3: Importing required packages

Import the required packages in your Python script

from pytube import YouTube
import os
import streamlit as st
from streamlit_extras.add_vertical_space import add_vertical_space

Step 4: build a basic Streamlit app front-end

Add the following code to your python script

with st.sidebar:
    st.title('YouTube Downloader')

    add_selectbox = st.sidebar.selectbox(
    "Select what you want to download?",
    ("Video", "audio"))

    st.markdown ('''
    ## About 
    You can now download your favorite media from YouTube by just providing the YouTube link to it;
    and selecting to download the video 
    or audio file of the media provided:

    - [streamlit](https://streamlit.io)             
    - [pytube](https://pytube.io/)             

    ''')

    add_vertical_space(5)
    st.write('Made by [Dev Tobs](https://twitter.com/AnyigorTobias)')

def Download():
    #add radio buttons and modify codes to allow user select the type of file needed





if __name__ == '__main__':
    Download()

Open a new terminal and navigate to your project directory.
Open your already existing virtual environment in the new terminal by entering the following code
pipenv shell
Run the Streamlit application you just built by entering this code in the command line interface (CLI)

streamlit run "name of your python script".py

If your Python script is named 'youtube.py' then you have to run

streamlit run youtube.py

this will bring up the following in your browser.

Step5: Download YouTube videos using pytube

Add the following to the python script to enable users download youtube videos through it





def Download():
    #add radio buttons and modify codes to allow user select the type of file needed
    # this will add a header to the streamlit app
    st.header("Youtube downloader")
    if add_selectbox == 'Video':

        # this brings up an input box for the url
        youtube_url = st.text_input("Enter the YouTube URL")

        # the st.radio brings up selection buttons for selectiong video resolution
        genre = st.radio(
                    "Select the resolution you will like to download",
                    ["Highest Resolution", "720p", "480p", "360p", "240p", "144p"]
                    )
        # this brings up a download button you can click on to initiate video download
        if st.button("Download video"):
                try:
                    youtubeObject = YouTube(youtube_url)


                    if genre == "Highest Resolution":
                        youtubeObject = youtubeObject.streams.get_highest_resolution()
                    elif genre == "720p":
                        youtubeObject = youtubeObject.streams.get_by_resolution("720p")
                    elif genre == "480p":
                        youtubeObject = youtubeObject.streams.get_by_resolution("480p")
                    elif genre == "360p":
                        youtubeObject = youtubeObject.streams.get_by_resolution("360p")

                    else:
                        youtubeObject = youtubeObject.streams.get_by_resolution("144p")
                     # creates a directory for downloads   
                    if youtubeObject:  
                        save_dir = 'output/.mp4'
                        os.makedirs('output', exist_ok=True)
                        youtubeObject.download(output_path=save_dir)
                        st.success("Download completed successfully.")
                    else:
                        st.error("No suitable video stream found for the selected resolution, try another resolution")

                except Exception as e:
                    st.error(f"An error occurred: {e}")

Step6: Filter audios from YouTube Vidoes

Pytube has a method that makes it possible to download just audio of any YouTube content. Add the code below to your script to make that possible

 if add_selectbox == 'audio':
        youtube_url = st.text_input("Enter the YouTube URL")

        if st.button("Download audio"):

            try:
                file = ""

                youtubeObject = YouTube(youtube_url)
                audio = youtubeObject.streams.filter(only_audio = True).first()

                title = os.path.basename(file)[:-4]

                save_dir = 'output/.mp3'
                os.makedirs('output', exist_ok = True)
                file = audio.downloutput_path = save_dir)
                st.success("Download completed successfully")
            except Exception as e:
                st.error(f"An error occurred: {e}")

Here is the complete python script

from pytube import YouTube
import os
import streamlit as st
from streamlit_extras.add_vertical_space import add_vertical_space
with st.sidebar:
    st.title('YouTube Downloader')

    add_selectbox = st.sidebar.selectbox(
    "Select what you want to download?",
    ("Video", "audio"))

    st.markdown ('''
    ## About 
    You can now download your favorite media from YouTube by just providing the YouTube link to it;
    and selecting to download the video 
    or audio file of the media provided:

    - [streamlit](https://streamlit.io)             
    - [pytube](https://pytube.io/)             

    ''')

    add_vertical_space(5)
    st.write('Made by [Dev Tobs](https://twitter.com/AnyigorTobias)')

def Download():
    #add radio buttons and modify codes to allow user select the type of file needed
    # this will add a header to the streamlit app
    st.header("Youtube downloader")
    if add_selectbox == 'Video':

        # this brings up an input box for the url
        youtube_url = st.text_input("Enter the YouTube URL")

        # the st.radio brings up selection buttons for selectiong video resolution
        genre = st.radio(
                    "Select the resolution you will like to download",
                    ["Highest Resolution", "720p", "480p", "360p", "240p", "144p"]
                    )
        # this brings up a download button you can click on to initiate video download
        if st.button("Download video"):
                try:
                    youtubeObject = YouTube(youtube_url)


                    if genre == "Highest Resolution":
                        youtubeObject = youtubeObject.streams.get_highest_resolution()
                    elif genre == "720p":
                        youtubeObject = youtubeObject.streams.get_by_resolution("720p")
                    elif genre == "480p":
                        youtubeObject = youtubeObject.streams.get_by_resolution("480p")
                    elif genre == "360p":
                        youtubeObject = youtubeObject.streams.get_by_resolution("360p")

                    else:
                        youtubeObject = youtubeObject.streams.get_by_resolution("144p")
                     # creates a directory for downloads   
                    if youtubeObject:  
                        save_dir = 'output/.mp4'
                        os.makedirs('output', exist_ok=True)
                        youtubeObject.download(output_path=save_dir)
                        st.success("Download completed successfully.")
                    else:
                        st.error("No suitable video stream found for the selected resolution, try another resolution")

                except Exception as e:
                    st.error(f"An error occurred: {e}")


if add_selectbox == 'audio':
        youtube_url = st.text_input("Enter the YouTube URL")

        if st.button("Download audio"):

            try:
                file = ""

                youtubeObject = YouTube(youtube_url)
                audio = youtubeObject.streams.filter(only_audio = True).first()

                title = os.path.basename(file)[:-4]

                save_dir = 'output/.mp3'
                os.makedirs('output', exist_ok = True)
                file = audio.downloutput_path = save_dir)
                st.success("Download completed successfully")
            except Exception as e:
                st.error(f"An error occurred: {e}")



if __name__ == '__main__':
    Download()

I have made the codes available on Github, follow the steps in the readme file to run this in your local machine