Time Optimization

Roman — Thu, 29 Aug 2019 10:16:05 +0000

Why for?

From time to time I used to use some web projects, which helps me to find something out within the game I play (Ingress Prime).

So, recently I have started collection banners, images, with the missions.
To see the banners, I use Ingress Mosaic website.

To reach the SpecOps Onyx medal, I have to complete 500 missions, but to get in top - at least 1400. So, to count the missions and length I am using Laravel 5.8 with the CLI.

What I did?

Made an artisan command with the php artisan make:command
Installed dependency of fabpot/goutte to make an HTTPS Requests and work with the content efficiently
Learned the structure of the pages I need to work with
Go Code!

Firstly, I had to make an alias for the name of the command to be executed from the CLI. I've came up with the next shorthand: profile:parser {nickname}.
30 minutes - and I've came with the next code.

        /** Fetching profile page **/
        $this->http = new Client([
            'base_uri' => 'https://ingressmosaik.com'
        ]);

        /** Transforming the response to the DOM **/
        $dom = new Crawler($this->http->get("/profil/{$this->argument('nickname')}")->getBody()->getContents());
        $todoItems = $dom->filter('div#this_todo > div');

        $mosaics = [];

        /** Fetching no. of the mosaics **/
        $todoItems->each(function (Crawler $node) use (&$mosaics) {
            $link = $node->filter('h5 > a')->attr('href');
            list(, , $no) = explode('/', $link);
            $mosaics[] = intval($no);
        });

        $targets = [
            'cities' => [],
            'missions' => 0,
            'distance' => 0.00
        ];

        collect($mosaics)
            ->each(function (int $id) use (&$targets) {
                /** Fetching mission page **/
                $missionPage = new Crawler($this->http->get("/mosaic/{$id}")->getBody()->getContents());
                $title = $missionPage->filter('div.panel-heading-1 > h4');
                $block = $missionPage->filter('div#mo_img > div.panel-body-1');

                list($title,) = explode(' - ', $title->text());

                $city = $block->filter('div > span')->eq(1)->text();
                $mc = $block->filter('div > span')->eq(6)->text();
                $lt = $block->filter('div > span')->eq(7)->text();
                $startPortal = $missionPage->filter('div#portals > table')->eq(0)
                    ->filter('a')
                    ->eq(1);

                $title = trim($title);

                $this->output->text("Parsing '{$title}'");

                $targets['cities'][$city][] = [
                    'mosaic' => trim($title),
                    'first_portal' => $startPortal->attr('href'),
                    'missions' => intval($mc),
                    'distance' => floatval($lt)
                ];

                $targets['missions'] += intval($mc);
                $targets['distance'] += floatval($lt);
            });

        $targets['distance'] = (float)number_format($targets['distance'], 2);

        foreach ($targets['cities'] as $city => $mosaicsList) {
            $targets['cities'][$city] = collect($mosaicsList)
                ->sortBy('missions', SORT_DESC, true);
        }

        file_put_contents(public_path('missions.json'), json_encode($targets, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE));

Yes, this code is ugly, as all of it is in the handle() method. But it's a quick solution to calculate all to-do banners and save all data into the JSON file.

DEV Community: Roman

Time Optimization

Why for?

What I did?