DEV Community

Roman
Roman

Posted on

2 2

Time Optimization

Why for?

From time to time I used to use some web projects, which helps me to find something out within the game I play (Ingress Prime).

So, recently I have started collection banners, images, with the missions.
To see the banners, I use Ingress Mosaic website.

To reach the SpecOps Onyx medal, I have to complete 500 missions, but to get in top - at least 1400. So, to count the missions and length I am using Laravel 5.8 with the CLI.

What I did?

  • Made an artisan command with the php artisan make:command
  • Installed dependency of fabpot/goutte to make an HTTPS Requests and work with the content efficiently
  • Learned the structure of the pages I need to work with
  • Go Code!

Firstly, I had to make an alias for the name of the command to be executed from the CLI. I've came up with the next shorthand: profile:parser {nickname}.
30 minutes - and I've came with the next code.

        /** Fetching profile page **/
        $this->http = new Client([
            'base_uri' => 'https://ingressmosaik.com'
        ]);

        /** Transforming the response to the DOM **/
        $dom = new Crawler($this->http->get("/profil/{$this->argument('nickname')}")->getBody()->getContents());
        $todoItems = $dom->filter('div#this_todo > div');

        $mosaics = [];

        /** Fetching no. of the mosaics **/
        $todoItems->each(function (Crawler $node) use (&$mosaics) {
            $link = $node->filter('h5 > a')->attr('href');
            list(, , $no) = explode('/', $link);
            $mosaics[] = intval($no);
        });

        $targets = [
            'cities' => [],
            'missions' => 0,
            'distance' => 0.00
        ];

        collect($mosaics)
            ->each(function (int $id) use (&$targets) {
                /** Fetching mission page **/
                $missionPage = new Crawler($this->http->get("/mosaic/{$id}")->getBody()->getContents());
                $title = $missionPage->filter('div.panel-heading-1 > h4');
                $block = $missionPage->filter('div#mo_img > div.panel-body-1');

                list($title,) = explode(' - ', $title->text());

                $city = $block->filter('div > span')->eq(1)->text();
                $mc = $block->filter('div > span')->eq(6)->text();
                $lt = $block->filter('div > span')->eq(7)->text();
                $startPortal = $missionPage->filter('div#portals > table')->eq(0)
                    ->filter('a')
                    ->eq(1);

                $title = trim($title);

                $this->output->text("Parsing '{$title}'");

                $targets['cities'][$city][] = [
                    'mosaic' => trim($title),
                    'first_portal' => $startPortal->attr('href'),
                    'missions' => intval($mc),
                    'distance' => floatval($lt)
                ];

                $targets['missions'] += intval($mc);
                $targets['distance'] += floatval($lt);
            });

        $targets['distance'] = (float)number_format($targets['distance'], 2);

        foreach ($targets['cities'] as $city => $mosaicsList) {
            $targets['cities'][$city] = collect($mosaicsList)
                ->sortBy('missions', SORT_DESC, true);
        }

        file_put_contents(public_path('missions.json'), json_encode($targets, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE));

Enter fullscreen mode Exit fullscreen mode

Yes, this code is ugly, as all of it is in the handle() method. But it's a quick solution to calculate all to-do banners and save all data into the JSON file.

Heroku

This site is built on Heroku

Join the ranks of developers at Salesforce, Airbase, DEV, and more who deploy their mission critical applications on Heroku. Sign up today and launch your first app!

Get Started

Top comments (0)

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more

👋 Kindness is contagious

Please leave a ❤️ or a friendly comment on this post if you found it helpful!

Okay