Let's discuss the structure of our simple PHP application, folders, files, and functions and transform the project to grow it.
You and I have developed a small PHP application that makes an HTTP request to the GitHub API and stores the result in the database.
Step 1 - How to Develop a Simple Web Application Using Docker-compose, Nginx, PHP 8, and MongoDB 6.
Step 2 - How to Make HTTP Requests to API in PHP App Using GitHub API Example and Write to Percona Server for MongoDB.
Our code already performs different functions:
- Reading environment variables.
- Connecting to the database.
- Running API queries.
- Looping and writing to the database.
The code is already hard to fit on the screen, and it's time to think about dividing the code into files, folders, and functions.
Frameworks are usually responsible for separating code into folders and files. But we don't use frameworks, so we'll do it ourselves. At this stage, we will not make the structure too complicated. Our task is to learn how to change it so that we can change it at any time in the future.
I prefer to change the structure of files and folders as the project grows, grouping files according to meaning and logic. In this article, we'll make the first change, and we'll do them more times in the future.
Let's start optimizing our application.
Initializing and configuring the application
Usually, PHP scripts are executed sequentially starting with index.php, as in our case.
At the beginning of index.php, we connect the composer libraries, read environment variables, create a database connection, and create an object for HTTP requests. This will be required for each script. I propose to put this in a separate init.php file.
Create an app/init.php file and move the initialization to it. We simply move the code from the index.php file, leaving only the logic of the script.
app/init.php
<?php
// Enabling Composer Packages
require __DIR__ . '/vendor/autoload.php';
// Get environment variables
$local_conf = getenv();
define('DB_USERNAME', $local_conf['DB_USERNAME']);
define('DB_PASSWORD', $local_conf['DB_PASSWORD']);
define('DB_HOST', $local_conf['DB_HOST']);
// Connect to MongoDB
$db_client = new \MongoDB\Client('mongodb://'. DB_USERNAME .':' . DB_PASSWORD . '@'. DB_HOST . ':27017/');
$app['db'] = $db_client->selectDatabase('tutorial');
$app['http'] = new \GuzzleHttp\Client();
You may also notice that I created an array or $app object, and added HTTP and db as array elements. That way I can use $app anywhere to pass to functions and have HTTP queries and database handling there.
And in the index.php file itself, we simply include our init.php
_app/index.php
<?php
require __DIR__ . '/init.php';
dd($app);
And print db and http with dd() to make sure they work and are available in index.php.
Leave the rest of the code in index.php unchanged for now and run localhost in the browser.
Let's get to the functions
Functions are needed to group code that is executed multiple times.
Functions can be initialized either in the executable PHP file itself, next to the code, or in a separate file that includes, such as composer libraries. You can pass parameters, variables or arrays into functions and functions can return some result.
For example, we make a GET HTTP request to the GitHub API at a certain URL. We will probably need to make another type of request to another URL, but it will still be an HTTP request using guzzle.
I assume in advance that I will have many different functions: general, for GitHub, for the database. So let's create a func folder in the app folder.
And in the app/func folder, create a github.php file. Create our first function there that will request the GitHub API.
We'll pack the code responsible for the HTTP request into a function.
app/func/github.php
<?php
function fn_github_api_request($app, $url, $method, $params = [])
{
try {
$response = $app['http']->request($method, $url , [
'query' => $params
]);
$result = $response->getBody();
$result = json_decode($result, true);
} catch (GuzzleHttp\Exception\ClientException $e) {
$response = $e->getResponse();
$responseBodyAsString = $response->getBody()->getContents();
echo $responseBodyAsString;
}
if (empty($result)) {
$result = false;
}
return $result;
}
A little clarification, we will modify this file in the future, the first function is not the best. You should understand that you will need to change frequently, changing parameters and variables within functions.
Now go back to the index.php, and connect our first function file.
app/index.php
<?php
// Enabling Composer Packages
require __DIR__ . '/init.php';
require __DIR__ . '/func/github.php';
$url = 'https://api.github.com/search/repositories';
$params = [
'q' => 'topic:mongodb',
'sort' => 'help-wanted-issues'
];
// New function
$repositories = fn_github_api_request($app, $url, 'GET', $params);
if (!empty($repositories['items'])) {
foreach($repositories['items'] as $key => $repository) {
$updateResult = $app['db']->repositories->updateOne(
[
'id' => $repository['id'] // query
],
['$set' => $repository],
['upsert' => true]
);
}
}
dd($repositories);
Run localhost and see the same result. Our request was executed, and we got the list of repositories.
The structure of our code in the app folder will look like this.
app/
.
├── func
│ └── github.php
├── vendor
├── composer.json
├── index.php
└── init.php
Loops and pagination
All this time we made only one API request and got the same number of repositories.
I propose to run our repository retrieval function in a loop and get 1000 repositories.
We will use the for loop
for ($i =1 ; $i <= 30; $i++) {
// Do something
}
Where i will be the page number for request to api.
app/index.php
<?php
// Enabling Composer Packages
require __DIR__ . '/init.php';
require __DIR__ . '/func/github.php';
$url = 'https://api.github.com/search/repositories';
$params = [
'q' => 'topic:mongodb',
'sort' => 'help-wanted-issues'
];
for ($i = 1; $i <= 30; $i++) {
$params['page'] = $i;
$repositories = fn_github_api_request($app, $url, 'GET', $params);
fn_github_save_repositories($app, $repositories);
echo "Page: " . $i . " ";
}
I converted the index.php file so that:
- The loop will add the page parameter corresponding to i and execute a request to the api.
- I also made a function fn_github_save_repositories in which I placed the code responsible for saving to the database.
- I added the output of the page sequence number.
When I ran localhost in the browser and looped, I saw that 9 requests were executed, and then I got an error
Page: 1 Page: 2 Page: 3 Page: 4 Page: 5 Page: 6 Page: 7 Page: 8 Page:
9 Page: 10 {"message":"API rate limit exceeded for *.*.*.*.
(But here's the good news: Authenticated requests get a higher rate limit.
Check out the documentation for more details.)",
"documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}
Page: 11 {"message":"API rate limit exceeded for *.*.*.*.
(But here's the good news: Authenticated requests get a higher rate limit.
Check out the documentation for more details.)",
"documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}
We made requests to the API as unauthorized users and got used to the limits. We need to add authorization. We'll fix that in the next article, as adding authorization will require a few big steps.
Check our database with MongoDB Compass
I ended up with 293 repository documents in the database.
Conclusion
As a result of this modification, we learned how to split one script into several scripts and create functions. We did a lot of queries and increased our database. True, we now know that API has limits.
You can check and run the source code in the repository.
In the next post, we will continue to modify and improve our application. We will add new functions and features. I'm sure we'll end up with a very useful app, and you'll learn how to develop it.
Ask me questions, I'll be happy to answer them.
Top comments (4)
Thank you for this article!
I always wonder what's the best way to organise my projects so I'm happy to see more ideas about it.
P.S. - I've noticed you wrote 2 articles prior to this one that are related to its subject.
I know dev.to has a 'series' feature, did you try to use it?
dev.to/kallmanation/dev-to-writing...
Thank you, yes, I think I need to start using the series functionality.
The structure is a complex thing, the project I describe now will have a few more transformations, as will be added:
It will be like our own framework. :)
Sounds really interesting!
I admit I'm a huge fan of open source systems & frameworks, so it's nice seeing what you take into consideration when developing this one.
Just started following you to get a glimpse into the process :)
True, a sample of optimized shape models written in Php could be found here at github.com/Adesoji1/Shape-Data-Model