DEV Community: Jakub Krajniak

Convert Google Slides to PDF

Jakub Krajniak — Thu, 19 Sep 2024 19:35:28 +0000

How to convert Google Slides to PDF? Well, just open Google Slides, then click in menu File -> Downdload -> PDF Document. That's all ;)

This sounds easy, but what if you cannot open the slides in Google Slides? What if you only have the embedded interface, the one that is created when you click File -> Share -> Publish to web? In fact, there is no easy solution. You cannot print; it just results in a blank, white page. There is also no download option from the interface.

In this short article, I will show you how to solve this problem by using a simple Python script.

Ingredients

If we cannot download the PDF from the slides or print it, we can still take screenshots ;) The only thing is that we don't want to do this manually. Instead, we'll just use the tool that is used to perform tests - Selenium.

Selenium is a tool that allows you to automate your web application tests. It uses webdrivers, which are headless versions of web browsers. The interactions with the web, like clicking, inputting text, etc., are coded using e.g., Python or other supported languages. Here, I will use Python.

Code

The main function is convert_slides_to_png, which takes the URL as input and stores the screenshots in the specified directory.

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from argparse import ArgumentParser


def convert_slides_to_png(url: str, file_prefix: str, output_dir: str):
    service = Service('/usr/local/bin/chromedriver')  # Replace with the actual path
    options = Options()
    options.add_argument("--disable-search-engine-choice-screen")
    options.add_argument("--headless=new")  # Run in headless mode (no visible browser window)
    driver = webdriver.Chrome(service=service, options=options)
    driver.set_window_size(1492, 1119)
    driver.get(url)  # Replace with your desired URL

    WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.TAG_NAME, 'body')))

    # Find the element to hide (replace with your element locator)
    driver.execute_script("""
        var elements = document.querySelectorAll('.punch-viewer-navbar');
        for (var i = 0; i < elements.length; i++) {
            elements[i].style.display = 'none';
        }
    """)

    def get_page_from_url(url: str) -> int:
        params = url.split('?')[1]
        params = params.split('=')
        page_id = int(params[1].split('.')[1].replace('p', ''))
        return page_id


    body_element = driver.find_element(By.TAG_NAME, 'body')
    body_element.send_keys(Keys.END)

    new_url = driver.current_url
    end_page_num = get_page_from_url(new_url)

    body_element.send_keys(Keys.HOME)

    for i in range(0, end_page_num):
        file_path = f'{output_dir}/{file_prefix}_page_{i}.png'
        driver.save_screenshot(file_path)
        print(f'Page {i} of {end_page_num} saved to {file_path}')
        body_element.send_keys(Keys.ARROW_RIGHT)

    print('Done')

Here, we actually don't click on the prev/next buttons; instead, we simulate pressing keys (HOME, ARROW_RIGHT, END) to interact with the Google Slides interface.

Let's just add a simple CLI, and we are ready to get the screenshots.

def main():
    parser = ArgumentParser(description='Convert Google Slides to PNG')
    parser.add_argument('--url-file', type=str, help='File with urls of the Google Slides')
    parser.add_argument('--output-dir', type=str, default='slides', help='Output directory')
    args = parser.parse_args()

    with open(args.url_file, 'r') as f:
        urls = f.readlines()
        print(f'Processing {len(urls)} urls')
        for url in urls:
            print(f'Processing {url}')
            url, file_prefix = url.strip().split(',')
            convert_slides_to_png(url, file_prefix, args.output_dir)

The list of slides is provided as a CSV file with columns: url and file_prefix.

https://docs.google.com/presentation/d/1rTqxnE6_IrTVF8xjYz_ejdrFWbgdjCJgVLD34PVgifw/embed;presentation1

To run the script, you need to

$ python3 cli.py --url-file urls.txt --output-dir slides

This will read URLs from urls.txt and output slides to slides/, where each slide page will be prefixed by presentation1.

Happy scrapping!

If you liked the post, then you can buy me a coffee. Thanks in advance.

FOSDEM 2024: What You Need to Know

Jakub Krajniak — Tue, 06 Feb 2024 16:22:23 +0000

Just back from the bustling FOSDEM 2024 in Brussels, here's some practical information to help you navigate this annual open-source conference effectively.

Be Prepared for Crowds

FOSDEM is free to attend, so expect large crowds. This means trams might be full, talks might have limited seating, and there might be queues for food and coffee. Consider it part of the open-source community's inclusivity.

Planning is Essential

With 61 concurrent tracks, planning your schedule in advance is crucial. While attending multiple tracks might seem appealing, focusing on a few will allow you to learn more efficiently and avoid rushing between buildings. Remember, the ULB campus is large, so factor in travel time.

Tips for Attendees

Plan effectively: Use the online schedule to create a focused schedule within 2-3 tracks you're interested in.
Stay energized: Start your day with a substantial breakfast to keep you going through sessions and potential waiting times.
Avoid coffee queues: Bring a reusable mug and your preferred coffee to skip the inevitable waiting line for coffee. Saving time means more learning and networking opportunities.
Choose convenient accommodation: Consider staying near ULB, ideally with good public transport options or within walking distance. This minimizes travel time and maximizes conference time.
Wear a mask: Due to the crowds, bringing and wearing a mask in common areas and talks is recommended, especially if you're concerned about illness. I've caught immediately a cold, nothing nice.

So, was attending FOSDEM 2024 worth it?

It depends.

Sure, all the lectures are streamed online with live chat, so physically being there isn't strictly necessary. But, as with many online experiences, something gets lost. Being in the room forces you to focus, take notes by hand, and even participate. No laptop distractions, just pen and paper engagement.

Then there's the social aspect. Connecting with fellow developers, project contributors, and potential collaborators is a huge part of the event, and that's something you miss online.

Ultimately, the decision comes down to your priorities. If convenience and affordability are key, online attendance is great. But if you crave energy, engagement, and social connections, the in-person experience can be truly valuable.

If you liked the post, then you can buy me a coffee. Thanks in advance.

How to debug your LAMMPS simulation using CLion

Jakub Krajniak — Sat, 06 Mar 2021 18:36:51 +0000

From time to time I need to debug LAMMPS code to find out why a simulation does not work as expected. The easiest way is to jump into the runtime using a debugger. Although gdb does the job, it is not the most intuitive and easy to use the tool.

In my daily work, whenever I can, I use JetBrains toolbox, for C++ programming - CLion. It definitely simplifies work, allowing as well to easy debug the code.

LAMMPS uses CMake to control the build process. Fortunately for us, CLion integrates with CMake projects very well.

So let's start.

Setup project

Open CLion, if you end up in the Welcome Window then click on Open and select the root of LAMMPS directory.

If you end up in the main window with an already opened project, you can go to Open ➡️ File ➡️ Open... and select the root of LAMMPS directory.

2. To enable CMake project, open cmake/CMakeLists.txt. You should see in the editor window on the top link to Load CMake project, click on it.

That is it, now you can compile your LAMMPS code directly from CLion, run and debug.

Enable/disable different compilation options

LAMMPS contains multiple packages, which enables various of capabilities. This modular design allows compiling the final binary executables with only these functions that are needed to simulate your system.
You can enable/disable features and settings by -D argument. The other, more convenient option, is to use CMake presets, which contains predefined variables.

The preset can be enabled from command line:

$ mkdir build; cd build
$ cmake -C ../cmake/presets/minimal.cmake ../cmake

and to enable multiple one at the same time

$ cmake -C ../cmake/presets/nolib.cmake -C ../cmake/presets/gcc.cmake -C ../cmake/presets/most.cmake  ../cmake

You can attach the same settings in the Clion project configuration by putting CMake command line option in CMake options field.

Run your simulation script

Now, when we set up our project, we can run our simulation script.

The easiest way is to make a copy of an existing configuration and include the simulation script.
Go to Run ➡️ Edit Configurations...

In the list of targets, search for lmp and click on Copy icon, change the name of the target.
Next, set the Working directory to the path where your simulation scripts are located. In the Program arguments you can specify which simulation script to run, e.g. -in in.lmp if your simulation script is in in.lmp file.

Run debugger

Running debugger is pretty simple. In the main window select the target to run, and click on the Debug icon.

I have guided you step by step on how to set up debugging for LAMMPS package. However, you can debug your GROMACS or ESPResSo++ simulations.

Happy debugging!

If you liked the post, then you can buy me a coffee. Thanks in advance.

Watch your stream

Jakub Krajniak — Thu, 10 Sep 2020 15:55:54 +0000

In the last story, I have discussed how to observe distributed cloud computation. Here, I will focus on a bit similar topic. Mainly, how to observe your stream of data flowing into your system, precisely how to decide that at a certain moment there won't be any more data flowing, and we are ready to do the post-processing.

This perhaps sounds in contradiction to the common approach in dealing with streams, where one could expect to process elements one-by-one. But let us consider the following situation:

An initial event invokes a set of producers to push data into a stream;
Producers simultaneously push data to the stream. Let then assume that the consumer stores the records in some permanent storage like S3;
Only after all expected data are in permanent storage, we can run the next transformation, e.g. some aggregations;

A very simple approach would be to track the execution of the producers (e.g. using the method which I showed in my previous story). The third step will be executed only if all of the producers finish computations.

The major drawback of this approach is not taking into account any delays related to passing records through the stream and storing them in permanent storage. This could, in the end, leads to computing aggregations on a partially completed data set.

A better idea could be to use an observer to track the number of records that are stored permanently in the storage.
Let us consider the following architecture, using a common component from AWS: Kinesis Firehose, S3, Lambda, DynamoDB.
The overall diagram is shown below

We have a bunch of lambdas which produce some data and push them to Kinesis Firehose. This is then stored in S3 bucket.
A monitor lambda is triggered whenever Firehose creates a new object in S3 bucket. We count such events and update the counter in DynamoDB table. The primary key of the record is the observed bucket name.

A prototype handler for such event can be defined as follows:

import boto3
import os

dynamodb = boto3.resource('dynamodb')

observerTableName = os.environ.get('observerTableName')
table = dynamodb.Table(observerTableName)


def monitor(event, context):
    for record in event.get('Records', []):
        if record.get('eventName', '') == 'ObjectCreated:Put':
            bucketName = record['s3']['bucket']['name']
            table.update_item(
                Key={'id': bucketName},
                UpdateExpression='ADD num_records :val',
                ExpressionAttributeValues={':val': 1}
            )

A second part of the diagram is the observer.
This lambda uses a SQS queue to run recurrently and reads the content of the storage counter DynamoDB table.
The algorithm behind the observer is quite simple and can be described by the following diagram

It can be implemented as follows

def observer(event, context):
    for record in event['Records']:
        payload = record['body']
        message = json.loads(payload)
        if message.get('repeated', 0) >= MAX_NUM_REPEAT:
            # call external service, ready to handle the data in storage.
            print(f'{message=} finished - calling external service')
            continue

        res = table.get_item(Key={'id': message['bucket']})
        if 'Item' in res:
            item = res['Item']
            num_records = int(item['num_records'])
            if num_records == message['last_num_records']:
                message['repeated'] += 1
            else:
                message['repeated'] = 0  # Reset the repeat counter.
            message['last_num_records'] = num_records
            sqs.send_message(QueueUrl=os.environ['selfSQSURL'], MessageBody=json.dumps(message), DelaySeconds=30)

with the SQS message in the following structure

{
  "repeated": 0,
  "last_num_records": 0,
  "bucket": "bucket name"
}

Two parameters have to be adjusted. The first is the length of the observation window (declared in the above code as MAX_NUM_REPEAT). The second parameter is the delay between reads from the DynamoDB table, here set to 30 seconds.

Let me comment on these two parameters.
If the producer is a slow process, and we sample too fast (with short delay time), then we can falsely consider the process to be finished.
On the other hand, if the producer is a fast process, we can unnecessarily wait MAX_NUM_REPEAT * delay seconds before the observer sends a notification that the data are ready.

You can use different optimization strategies for the delay time:

divide initial delay time by the repeated counter

delay = 30 if repeated == 0 else int(30 / repeated)

using exponential function

delay = numpy.ceil(30*numpy.exp(-repeated)).astype(int)

It depends on the nature of the production process which parameters are appropriate.

The method I have presented here can be useful in data processing, where we need to run some post-processing tasks after we have a dataset ready in the storage (like S3, Elasticsearch, etc.).
It needs some tuning to be applicable, and one has to consider as well, how to deal with any false-positive cases.

On the cover image, the Dijle river near Leuven, Belgium

If you liked the post, then you can buy me a coffee. Thanks in advance.

Efficient message broadcasting with SNS filtering

Jakub Krajniak — Thu, 20 Aug 2020 07:34:05 +0000

Broadcasting messages from one source to multiple destinations (or consumers) is a commonly seen pattern in distributed computing solutions.
A very simple case is when we would like to trigger multiple workers at the same time to run a parallel job. The trigger can be executed, e.g., as a CloudWatch cron-like event.

Here, the same message is delivered to multiple consumers. In terms of a specific implementation, using AWS cloud components, the above diagram can be transformed into the following architecture diagram:

Now, let say that we would like that pattern an event bus, on which messages to different consumers are published into the same communication channel, as depicted on the figure below.

The communication channel could be, e.g. SNS topic, and the consumers, as in the previous example, can be an AWS Lambdas. Multiple producers (marked with different colours) push messages to the communication bus and the consumers read the messages from it.

This solution has one major drawback, the same message will be delivered to multiple consumers, even if the consumer cannot handle that message. So for example, an orange consumer will receive message marked with green, or red. In principle, nothing bad could happen if the consumers can filter out such messages. However, this will involve additional computational time and cost.

We could think about several solutions to this kind of problem. The filtering, as said before, can be delegated to the consumers. There could be multiple communication channels, one per every message type, so the consumers can be linked directly with the appropriate channel. But on the cost of complicated architecture.

Instead of making the architecture complex, or move the filtering to the consumer code, we could use message filtering available in SNS service. By this, the consumers can get only a subset of the messages that are pushed to the topic.

The filtering is done on the attributes and values of the attributes assigned to the published message. A very basic example, using a serverless framework, is shown below

functions:
  hndl1:
    handler: handler.fun1
    events:
     - sns: 
          arn: !Ref NotifyTopic
          topicName: notifyTopic
          filterPolicy:
            messageType:
              - hndl1
            messageContext:
              - running

Here, the lambda hndl1 will be triggered by an incoming message on SNS topic only if the message will have two attributes: messageType, messageContext of values hndl1, running respectively.

Currently, SNS policy filtering can handle only attributes of types: String, String.Array and Number.
The filter values can be either matched exactly or treated as prefixes. In the previous example, the values were matched exactly. In the following example, we use keywords prefix to indicate that the value should be treated as a string prefix. Moreover, the keyword anything-but allows defining the values the attribute should not have (blacklist).

functions:  
  hndl2:
    handler: handler.fun2
    events:
      - sns: 
          arn: !Ref NotifyTopic
          topicName: notifyTopic
          filterPolicy:
            messageType:
              - hndl2
            messageContext:
              - incontext
              - prefix: type_
            messageStage:
              - anything-but:
                  - restart

It is worth to mention that prefix and constant values can be joined together. In the above example, messageContext matches either with incontext value or with type_1, type_2, type_b, type_abc … etc.
The same does not hold for anything-but keyword, which cannot be mixed with exact values.

Apart from working with strings, filters can also operate on numerical values. This is done by using keyword numeric, as in the example below

functions:
  hndl3:
    handler: handler.fun3
    events:
      - sns: 
          arn: !Ref NotifyTopic
          topicName: notifyTopic
          filterPolicy:
            messageType:
              - hndl3
            cost:
              - numeric: 
                  - ">"
                  - 0

Here, the message with attributes messageType equal to hndl3 and cost greater than zero. The numeric filter accepts the following operators: =, >, >=,<, and <=. In addition, you can define a range of values. In the example below, the value of cost attribute has to be in the range (0, 200) to be accepted.

cost:
  - numeric: 
      - ">"
      - 0
      - "<"
      - 200

You can even mix numeric and string attribute values, as in the example below

cost:
  - unknown
  - numeric:
      - "="
      - 100

where the cost attribute has to be either 100 or unknown.

SNS filtering can help you distribute the messages to right consumers, reducing your computational cost on a little price of writing filters.
It is quite robust tool and can be used for all kind of receiver the SNS subscription can handle.

If you liked the post, then you can buy me a coffee. Thanks in advance.