An Intelligent Transportation System using Self-driving Cars and Mobile Phones

Mahmoud Youssef — Sun, 24 May 2020 15:47:48 +0000

Overview of BARQ

Barq is an intelligent transportation system using self-driving cars and mobile phones. It makes it safe and easy for people and things to get where they are going. The Project consists of 4 essential parts :

Web Server
Admin Dashboard
Mobile Application
Self-Driving Car

Web Server

It is responsible for:

Handling mobile application and admin dashboard HTTP requests.
Finding the shortest path between user's location and car's location.
Monitoring car's locations and sending orders.
Tracking car's locations and sending it to admin dashboard and mobile application.
Dealing with database.

Admin Dashboard

It is responsible for:

Managing notifications and sending them to users using the mobile application.
Managing packages so user can register in one package to get more offers.
Managing coupons.
Managing companies and related history to each company.
Managing cars.
Managing issues.
Managing users.

Mobile Application

It provides more facilities to user:

Getting new notifications.
Getting new offers.
Making rides.
Adding credit cards.
Transferring money from credit cards to his wallet.
Reporting issues.
Asking for help via chat with customer service.
Viewing ride history.
Submitting codes for getting offers.

Self-Driving Car

It's responsible for:

Low level motion control.
Obstacle avoidance during motion.
Path tracking to reach destination.
Localization
Sensor Fusion using sensors besides robot odometry to increase accuracy of localization process.

Content

Introduction to Intelligent Transportation Systems
System Components and Architecture
Business Scenarios
Link to Code & Demos
Conclusion and Future Work

Introduction to Intelligent Transportation Systems

Self-driving car technology is advancing every day, and it’s only a matter
of time before fully driverless vehicles appear on public streets. Almost daily, there’s a new development in the driverless car space, and nearly every major car manufacturer, ride-sharing service and tech company from Apple to Google has bought into the driverless car industry. Self -driving car can be used in various and varied applications. The most common one is transferring someone from one place to another. It also can be used on indoor and outdoor delivery system to deliver items to the one who requested them. Here we are using it with BARQ to deliver people from place to place.

System Components and Architecture

Robot Platform

Redcat Racing Volcano EPX Electric Truck, Blue/Silver, 1/10 Scale
4WD Monster truck w/ Large monster truck tires, Lightweight molded plastic chassis, (8) Aluminum capped oil filled shocks, front and rear gear differentials.
Tunable suspension, Adjustable ride height, Tunable shocks, Tunable gear differentials, Adjustable camber, Adjustable toe angle. Speed Control: 40a Brushed Waterproof ESC
Aggressive tires, Independent suspension, Low center of gravity chassis for high speed performance, Ball bearings throughout.
19T brushed 550 electric motor with heat sink, Waterproof ESC, Waterproof high torque steering servo, 2.4GHz radio system.
Assembled & Ready to Run - Includes 7.2V 2000mAh NiMH Battery & NiMH Charger - Requires 4-AA Batteries for Transmitter.
Link: https://www.amazon.com/gp/product/B00CEP8YN4/ref=ox_sc_mini_detail?ie=UTF8&smid=ATVPDKIKX0DER&th=1

Motor Drivers

Motor drives are circuits used to run a motor. In other words, they are
commonly used for motor interfacing. These drive circuits can be easily interfaced
with the motor and their selection depends upon the type of motor being used and their ratings (current, voltage).
In our project we use two BTS7960 KIT having the following architecture:
• High-power drive full H-bridge driver module with thermal over-current
protection. High-current 43A.
• Double BTS7960 large current (43 A) H bridge driver.
• 5V isolate with MCU, and effectively protect MCU.
• 5V power indicator on board.
Page 19
CHAPTER 2. SYSTEM COMPONENTS AND ARCHITECTURE
• Voltage indication of motor driver output end.
• Heat sink can be added.
• Isolation chip 5 V power supply (can share with MCU 5 V).
• Size: 4 * 5 * 1.2 cm.
• Ability to reverse the motor forward, two PWM input frequency up to 25kHZ.
• Two heat flow passing through an error signal output.
• Isolated chip 5V power supply (can be shared with the MCU 5V), can also
use the on-board 5V supply.
• The supply voltage 5.5V to 27V.
Both drivers are powered by LIPO 7.4V -4200mAh battery.

Raspberry Pi

A Raspberry Pi is a credit card-sized computer originally designed for
education, inspired by the 1981 BBC Micro. Creator Eben Upton’s goal was to
create a low-cost device that would improve programming skills and hardware
understanding at the pre-university level. But thanks to its small size and accessible
price, it was quickly adopted by tinkerers, makers, and electronics enthusiasts for
projects that require more than a basic microcontroller. The Raspberry Pi is slower
than a modern laptop or desktop but is still a complete Linux computer and can
provide all the expected abilities that implies, at a low-power consumption level.
In our project we use Raspbery Pi 3 model B having the following architecture:
• Quad Core 1.2GHz Broadcom BCM2837 64bit CPU.
• 1GB RAM.
• BCM43438 wireless LAN and Bluetooth Low Energy (BLE) on board.
• 100 Base Ethernet.
• 40-pin extended GPIO.
• 4 USB 2 ports.
• 4 Pole stereo output and composite video port.
• Full size HDMI.
• CSI camera port for connecting a Raspberry Pi camera.
• DSI display port for connecting a Raspberry Pi touchscreen display.
• Micro SD port for loading your operating system and storing data.
• Upgraded switched Micro USB power source up to 2.5A.[14]

Sensors

1. IMU

An inertial measurement unit (IMU) is an electronic device that measures and
reports a body’s specific force, angular rate, and sometimes the orientation of
the body, using a combination of accelerometers, gyroscopes, and sometimes
magnetometers.
In Our project we use Adafruit 9-DOF IMU module having the following architecture:
• Dimensions: 20mm x 27mm x 4mm / 0.8” x 1.1” x 0.2”
• Header holes begin 4mm from the mounting holes
• Mounting Hole dimensions: 20mm x 12mm apart
• Uses I2C address 0x28 (default) or 0x29
• Weight: 3g

2. GPS

The Global Positioning System (GPS), is a satellite-based radionavigation
system owned by the United States government and operated by the United
States Air Force. It is a global navigation satellite system that provides
geolocation and time information to a GPS receiver anywhere on or near
the Earth where there is an unobstructed line of sight to four or more GPS
satellites. Obstacles such as mountains and buildings block the relatively
weak GPS signals.
In Our project we use DIYmall NEO-6M GPS module having the following architecture:
• Standalone GPS receiver
• U-blox NEO-6M GPS module
• Under 1 second time-to-first-fix for hot and aided starts
• SuperSense ® Indoor GPS: -162 dBm tracking sensitivity
• Anti-jamming technology
• Support SBAS (WAAS, EGNOS, MSAS, GAGAN)
• u-blox 6 50 channel positioning engine with over 2 million effective
correlators
• Timepulse
• 5Hz position update rate
• Operating temperature range: -40 TO 85°C
• UART TTL socket
• EEprom to store settings
• Rechargeable battery for Backup
• Build in 18X18mm GPS antenna
• RoHS compliant

3. LIDAR

Lidar is a surveying method that measures distance to a target by illuminating
the target with pulsed laser light and measuring the reflected pulses with
a sensor. Differences in laser return times and wavelengths can then be
used to make digital 3-D representations of the target. The name lidar,
now used as an acronym of light detection and ranging (sometimes light
imaging, detection, and ranging), was originally a portmanteau of light and
radar. Lidar sometimes is called 3D laser scanning, a special combination
of a 3D scanning and laser scanning. It has terrestrial, airborne, and mobile
applications.
In Our project we use YDLIDAR Lidar F4 2D Lidar scanner having the following architecture:
• Application field:Environment Scanning, SLAM Application and Robot
Navigation
• 360-degree laser rangefinder, Scanner range 12 Meter,MAX. 6000Hz
Ranging Sample rate
• OS:Windows, Android, ROS and Linux,Ultra long working life,Weight:
189g
• F4 uses Industrial Brushless Moto,Scanning Rate: 6000tims/s,Laser
grade: Class 1,Configurable Scanning Frequency: 5-12Hz
• Quick Start-F4 has complete drivers as well as complete SDK ,API and
documentation. It’s easy for users to get started quickly.

4. Webcam

A webcam is a small digital video camera directly or indirectly connected to
a computer or a computer network.
In Our project we use 1080P Nano Shield N920 Webcamera having the following architecture:
• Premium auto focus and light correction.
• Full HD 1080P at 30 fps recording.
• Noise cancelling microphone.
• Tripod ready mount.
• IM compatibility.

Software Components

Web Server

The aim of this part is that it can process requests and delivers data from and to clients, admins and cars over the internet or a local network.

Mobile Application

We use the mobile application for the main purpose of making rides and moving from one place to another via the mobile application.

Admin Dashboard

It's the control panel of the admins which provides flexible UI to manage resources.

Robot Software Platform

We integrate all components of the robot platform using Robot Operating
System (ROS). The aim of this part is to provide low-level device control, connect all processes together and provide message-passing between processes with no effort.

System Components and Architecture

To achieve its goal, our project is divided into several main modules
which are integrated together in an appropriate manner, these modules are as
follows:

Web server
Path planning
Main controller
Convolutional Neural Network
Classical control
Obstacle avoidance
Localization
Mobile Application
Admin Dashboard

These modules work on data from IMU, GPS, LIDAR, Webcam and Admin Dashboard then send actions to perform the tasks required by the client.

Whole system architecture:

Self-Driving Car architecture:

1. Web Server Components

The Web Service process resides on the web server. A web server is server software, or hardware dedicated to running this software, that can satisfy client requests on the World Wide Web. A web server can, in general, contain one or more websites. A web server processes incoming network requests over HTTP and several other related protocols. The primary function of a web server is to store, process and deliver web pages to clients. The communication between client and server takes place using the Hypertext Transfer Protocol (HTTP). Pages delivered are most frequently HTML documents, which may include images, style sheets and scripts in addition to the text content. We are using Node.Js Web Server

NodeJS Web Server

Node.js is an open-source, cross-platform, JavaScript runtime environment that executes JavaScript code outside of a web browser. Node.js lets developers use JavaScript to write command line tools and for server-side scripting—running scripts server-side to produce dynamic web page content before the page is sent to the user's web browser. Consequently, Node.js represents a "JavaScript everywhere" paradigm, unifying web-application development around a single programming language, rather than different languages for server- and client-side scripts.

Node.js operates on a single-thread event loop, using non-blocking I/O calls, allowing it to support tens of thousands of concurrent connections without incurring the cost of thread context switching. The design of sharing a single thread among all the requests that use the observer pattern is intended for building highly concurrent applications, where any function performing I/O must use a callback. To accommodate the single-threaded event loop, Node.js uses the libuv library—which, in turn, uses a fixed-sized thread pool that handles some of the non-blocking asynchronous I/O operations.
A thread pool handles the execution of parallel tasks in Node.js. The main thread function call posts tasks to the shared task queue, which threads in the thread pool pull and execute. Inherently non-blocking system functions such as networking translate to kernel-side non-blocking sockets, while inherently blocking system functions such as file I/O run in a blocking way on their own threads. When a thread in the thread pool completes a task, it informs the main thread of this, which in turn, wakes up and executes the registered callback.
A downside of this single-threaded approach is that Node.js does not allow vertical scaling by increasing the number of CPU cores of the machine it is running on without using an additional module, such as cluster, StrongLoop Process Manager, or pm2. However, developers can increase the default number of threads in the libuv thread pool. The server operating system (OS) is likely to distribute these threads across multiple cores. Another problem is that long-lasting computations and other CPU-bound tasks freeze the entire event-loop until completion.

Following are some of the important features that make Node.js the first choice.
• Asynchronous and Event Driven − All APIs of Node.js library are asynchronous, that is, non-blocking. It essentially means a Node.js based server never waits for an API to return data. The server moves to the next API after calling it and a notification mechanism of Events of Node.js helps the server to get a response from the previous API call.
• Very Fast − Being built on Google Chrome's V8 JavaScript Engine, Node.js library is very fast in code execution.
• Single Threaded but Highly Scalable − Node.js uses a single threaded model with event looping. Event mechanism helps the server to respond in a non-blocking way and makes the server highly scalable as opposed to traditional servers which create limited threads to handle requests. Node.js uses a single threaded program and the same program can provide service to a much larger number of requests than traditional servers like Apache HTTP Server.
• No Buffering − Node.js applications never buffer any data. These applications simply output the data in chunks.

ExpressJS

Express is a minimal and flexible Node.js web application framework that provides a robust set of features to develop web and mobile applications. It facilitates the rapid development of Node based Web applications. Following are some of the core features of Express framework −
• Allows to set up middlewares to respond to HTTP Requests.
• Defines a routing table which is used to perform different actions based on HTTP Method and URL.
• Allows to dynamically render HTML Pages based on passing arguments to templates.

Socket.io

Socket.IO is a JavaScript library for realtime web applications. It enables realtime, bi-directional communication between web clients and servers. It has two parts: a client-side library that runs in the browser, and a server-side library for Node. js. We 're using socket.io for communicating between the web server and cars, clients and admin dashboard.

Design Patterns

We worked using MVC Design Pattern (Model–view–controller) which is a software design pattern commonly used for developing user interfaces which divides the related program logic into three interconnected elements. This is done to separate internal representations of information from the ways information is presented to and accepted from the user. This kind of pattern is used for designing the layout of the page. But we didn't use the models for any purpose just for rendering some static web pages. We isolated our Models, Routes and Controllers.

MongoDB

MongoDB is a cross-platform, document-oriented database that provides, high performance, high availability, and easy scalability. MongoDB works on concept of collection and document. The main purpose of deciding to choose MongoDB:
• Document Oriented Storage − Data is stored in the form of JSON style documents.
• Index on any attribute
• Replication and high availability
• Auto-Sharding
• Rich queries
• Fast in-place updates
• Professional support by MongoDB

RESTful API's

A RESTful API is an application program interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data.
An API for a website is code that allows two software programs to communicate with each other. The API spells out the proper way for a developer to write a program requesting services from an operating system or other application.
A RESTful API -- also referred to as a RESTful web service or REST API -- is based on representational state transfer (REST), an architectural style and approach to communications often used in web services development.
REST technology is generally preferred over the more robust Simple Object Access Protocol (SOAP) technology because REST uses less bandwidth, making it more suitable for efficient internet usage.

API Testing

We used postman to test our endpoints. Postman is a collaboration platform for API development. Postman's features simplify each step of building an API and streamline collaboration so you can create better APIs—faster.

ER-Diagram

This is our Entity Relationship Diagram as a Database Design so we noticed a lot of issues that would face us. We searched about about best approaches and decisions made based on Referenced Documents in MongoDB and Embedded Objects.

2. Path Finding

The path planning module takes the source and destination locations and
generates the best path between them as several continuous nodes, then it sends the path to the main controller through the web server.

Challenges

The challenge here was we are building the graph using nodes and of course we can't cover all map using nodes near to each other, as it will include many nodes in the system. So we think to get user's location and then apply a range of distances that can be added and subtracted from user's location so it makes it easy for us instead of searching for all nodes in the graph which one is near to user's location to search among few nodes. But it is not the best solution. So we think about how can we do that in both x-y directions ( Something like Binary Search but in 2D ) and here we find better approach which is KD-Tree.

The Path finding depends on the source location and target location which the user determine when he makes a ride using the mobile app. But before getting the best path between those nodes we have to figure out from our graph which nodes are near to those nodes of the user located. So we apply first the KD-Tree as we can determine the nearest node to these locations then start our journey with the path finding algorithm.

For those who would like to know more about it you can see this link
over here: https://www.geeksforgeeks.org/k-dimensional-tree/

After getting the source and target nodes, we start getting the best path between them using A* Algorithm. A* (pronounced "A-star") is a graph traversal and path search algorithm, which is often used in computer science due to its completeness, optimality, and optimal efficiency. One major practical drawback is its {\displaystyle O(b^{d})}O(b^d) space complexity, as it stores all generated nodes in memory. Thus, in practical travel-routing systems, it is generally outperformed by algorithms which can pre-process the graph to attain better performance, as well as memory-bounded approaches; however, A* is still the best solution in many cases. For those who would love to know more about A* Algorithm: https://www.geeksforgeeks.org/a-search-algorithm/

Testing-Map

We designed a map using Photoshop as it would be the banner which the self-driving car would move on it as it's a real map. We generated our graph nodes and edges between streets as a real one.

3. Main Controller

The main controller takes in all the control actions from CNN, classical
control, obstacle avoidance modules and based on the current information provided from localization and path planning modules, it determines which control action will be sent to Arduino.

4. Convolutional Neural Network

The CNN (Neural Network) Module takes camera data as input and feeds
raw images into the CNN. The details of the CNN module that used in our system
is fully is discussed in chapter 5. After processing the image the module outputs the predicted control action corresponding to that image and sends it to the main controller using an open socket connection. CNN model is used only to follow lane where there is no intersections.

5. Classical Control

The classical control is used to generate actions at intersection points
based on current and next poses of the robot that is being known from path planning and sends it to the main controller.

6. Obstacle Avoidance

The Obstacle Avoidance module takes LIDAR data as input containing
ranges and distances, then if the distance is below a certain threshold it outputs a high-priority control action, that is sent to the main controller.

7. Localization

The localization module takes input: IMU, GPS and Odometry data, after
processing the data using “Sensor Fusion” and “Extended Kalman Filter” method,
which is fully discussed in chapter 5. It outputs an estimated location of the robot, that is sent to the main controller over a socket connection.

8. Mobile Application

We used React Native to develop our mobile application, It is an open-source mobile application framework created by Facebook. It is used to develop applications for Android, iOS, Web and UWP by enabling developers to use React along with native platform capabilities. Our mobile application has many features and screens which we talked about in the overview section above.

9. Admin Dashboard

This is a website for admins so they can manage the business resources: adding, removing and updating resources like packages, offers and viewing issues and bugs. They also can make reports based on data analysis that is viewed. We used Angular 9. Angular is a Javascript framework that is used by developers for building web, desktop, and mobile applications. Development of Angular applications involves usage of Typescript, which is a superset of Javascript, along with HTML, CSS etc. The code written in Typescript compiles to Javascript and is rendered in the browser. Angular is one of the most popular Javascript frameworks for building web applications.

Business Scenarios

The user download the mobile application then he can register after that we send the user a verification code via his phone number entered in registration. After finishing the registration phase he can modify his info section and add credit cards. He also can see the offers or notifications and register in a package which would give him more benefits. At the end the user can use finally the application to make rides by clicking Ride button.

The user clicks on ride button
The user chooses if he wants to go somewhere common suggested or special location ( given source and destination of the user location )
The web server handles the request from the client getting the source and destination locations.
The web server enables KD-Tree to get the nearest node to these locations.
The web server keeps monitoring car's location
The web server finds the shortest path between car's location and source location and between source and destination and send them to the car.
The web server inform the user of the time that the car will arrive in
The web server tells the admin that the car is accessed now by this user and keep sending it's location to the admin dashboard.
After car is arrived, the user can use it to move to the destination location.
The web server still keeps tracking of car's path as it will send the path to the admin dashboard for rides history.
The web server will inform the user that if the car arrived to the destination.
After finishing the ride the user can fill the feedback form.

Link to Code

1. Back End Repo:

Taxi-ai / Back-End

Barq - Back End

It is responsible for:

Handling mobile application and admin dashboard HTTP requests.
Finding the shortest path between user's location and car's location.
Monitoring car's locations and sending orders.
Tracking car's locations…

View on GitHub

2. Car Repo:

Taxi-ai / car-workspace

workspace for car contain all package.

Taxi-ai car catkin workspace

workspace for car contain all package.

current working nodes

camera publish node
lidar publish node
lane detection node

TODO

add IMU sensor node

code structure


* ydlidar "packge for our ydlidar"
* sensors "package to manage all sensors"
|   - msg "contain our custom type messages"
|   - scripts "our python files for publush and subscripe"
|   - launch "launch file to run nodes"
* perception "package contain all perception work."
|   - msg "contain our custom type messages"
|   - launch "launch file to run nodes"
|   - src "contain different perception methods"
|   |   - lane-detection
- README.md

Installation

OS X & Linux:

$ mkdir -p ~/catkin_ws/src
$ cd ~/catkin_ws/
$ catkin_make

Now that you have a workspace, clone or download this repo into the src directory of your workspace:

$ cd ~/catkin_ws/src
$ git clone clone https://github.com/Taxi-ai/car-workspace.git

install dep

…

View on GitHub

3. Dashboard Repo:

Taxi-ai / Barq-Dashboard

Barq -Dashboard

Dashboard Responsibilites:

Managing notifications and sending them to users using the mobile application.
Managing packages so user can register in one package to get more offers.
Managing coupons.
Managing companies and related history to each company.
Managing cars.
Managing issues.
Managing users

View on GitHub

1.Mobile App Demo

It's a recorded video uploaded with google drive:
https://drive.google.com/file/d/1uuzhDmIH8lHymYkIO1HDGTbixvxreiJ7/view

2. Dashboard Demo

We used the GitHub Student Package namepsace the we can host the admin dashboard:
https://barq.vision/dashboard

3. Web Server Demo

We used The GitHub Student Package Herou dynos to depoly our web server and database:
https://barq-api.herokuapp.com
https://barq-api.herokuapp.com/api/packages

4. Self-Driving Car Demo

It's a recorded video uploaded with google drive:
https://drive.google.com/open?id=147mQgc1XBfmoaSfXmVliH59qTHsQaIRm

Summarizing Tools & Technologies Used

NodeJs
ExpressJs
MongoDB
Postman
React-Native
Angular 9
Robot
Motor Driver
Raspberry Pi
IMU, GPS, Lidar and webcams sensors
ROS
CNN
PID
GitHub Student Package
Socket.io

Future Work

We decided to join the EED event next yaer, and we are going to try to work with real map like depending on Google Map API not only depending on specific map, However our algorithms works in general and not depending on the map but our capabilities and time weren't enough of course to learn how to apply this on a real map in a real world.

Additional Thoughts / Feelings / Stories

We decided that our graduation project should be something big, difficult and not even easy so we can at the end of the year learn many new things facing a lot of issues and difficulties and learn many new things over it. The idea is first came from Uber application but we decided to make it with a self-driving cars. We have been working a lot of hours each day, facing many issues and searching how to solve 'em and learning new things every single day. We believe that with taking challenges it would be much more helpful to ourselves in the career and this will help us to implement and develop what we have learnt throw years and still learning till this time. We believed that there is nothing called "hard" or "impossible" but with determination and passion you can learn and do a lot of things. Finally I am speechless, I don't know how to really tell my team that it was a great journey with them, having fun together, having troubles and difficult days of being disappointed. We have did a great job guys ! Yeaaah ❤ I want to thank everyone in my team, and of course without those brave heroes we couldn't make this graduation project to be like that.
Thank you everybody 😍

Ahmed Nassar : Working on the self-driving car
Mustafa Eldeeb : Working on the self-driving car
Mahmoud Samy : Working on the Dashboard
Alaa Fathalla : Working on the mobile application
Sara Ghoniem : Working on the mobile application

Finally it's a special thank to the one who always was behind our backs, helping us, motivating us and leading us "Eng.Mohamed Shehab" who's a real brother not just a Teaching Assistant and special thanks also for Dr.Mohamed Arafa who's our supervisor.

DEV Community: Mahmoud Youssef

An Intelligent Transportation System using Self-driving Cars and Mobile Phones

Overview of BARQ

Web Server

Admin Dashboard

Mobile Application

Self-Driving Car

Content

Introduction to Intelligent Transportation Systems

System Components and Architecture

Robot Platform

Motor Drivers

Raspberry Pi

Sensors

1. IMU

2. GPS

3. LIDAR

4. Webcam

Software Components

Web Server

Mobile Application

Admin Dashboard

Robot Software Platform

System Components and Architecture

1. Web Server Components

NodeJS Web Server

ExpressJS

Socket.io

Design Patterns

MongoDB

RESTful API's

API Testing

ER-Diagram

2. Path Finding

Challenges

Testing-Map

3. Main Controller

4. Convolutional Neural Network

5. Classical Control

6. Obstacle Avoidance

7. Localization

8. Mobile Application

9. Admin Dashboard

Business Scenarios

Link to Code

1. Back End Repo:

Taxi-ai / Back-End

Barq - Back End

2. Car Repo:

Taxi-ai / car-workspace

workspace for car contain all package.

Taxi-ai car catkin workspace

current working nodes

TODO

code structure

Installation

3. Dashboard Repo:

Taxi-ai / Barq-Dashboard

Barq -Dashboard

1.Mobile App Demo

2. Dashboard Demo

3. Web Server Demo

4. Self-Driving Car Demo

Summarizing Tools & Technologies Used

Future Work

Additional Thoughts / Feelings / Stories