February 26, 2021 • 6 min read

rideOS outperforms Google’s OR-Tools in Wait Time, Active Time, and Fleet Utilization

Executive Summary

  • The rideOS platform had a lower median wait time and active time compared to OR-Tools: 8.4 min vs 28.1 min, and 10.9 min vs 14.2 min respectively.
  • The rideOS platform had a higher median fleet utilization (i.e., % of time a vehicle has at least one passenger on board) compared to OR-Tools: 43.8% vs 37.7%, and comparable capacity utilization (i.e., how many passengers are in a vehicle).
  • Running a fleet on the rideOS platform will result in substantially better performance in terms of fleet efficiency and customer satisfaction.
    • While these simulations are run with passenger ride-hail data, the rideOS platform has successfully been used across a variety of use cases, including delivery and logistics, and thus we are confident that we can serve your needs.

Introduction

Google’s OR-Tools is a general solver that is able to solve a variety of problems, ranging from constraint programming, linear programming, traveling salesperson problems (TSPs), and vehicle routing problems (VRPs).

 

We were interested to compare our rideOS platform optimization against Google’s OR-Tools, for common use cases that come from our partners. In order to do so, we leveraged our Osiris Simulator platform to do an apples-to-apples comparison of our rideOS platform against Google’s OR-Tools.

 

Experimental Setup

Osiris abstracts running of a simulation from the optimization solvers through the use of adapters. Hence, for Osiris to connect to OR-Tools, we only had to write an adapter between the two systems. The figure below shows the experimental setup to compare rideOS’s platform optimization against Google’s OR-Tools.

 

osiris simulator

How the Osiris Simulator compares rideOS’s optimization against Google’s OR-Tools.

 

Since OR-Tools is a general solver, it can be applied to Vehicle Routing Problems (VRPs). In order for a VRP to be fully defined, an ETA matrix has to be provided that defines the travel times between any pair of locations in the VRP, e.g., the times to travel from origins to destinations. We call the rideOS platform’s ETA API to obtain this ETA matrix, which uses accurate real-time traffic information to generate the desired ETA matrix. 

 

The output from OR-Tools are plans for the vehicles, e.g., Vehicle 1 has to go to origin A, then origin B, then destination B, then destination A. However, the actual vehicle routes are not provided, e.g., the roads that the vehicle should take to go from origin B → destination B. In order to run the simulation, the simulated vehicles need fully instantiated routes. Thus, we take the plans created by OR-Tools, and call the rideOS platform’s Path API in order to generate routes that the vehicles take, which follow accurate real-time traffic.

 

When Osiris is run with the rideOS platform, we can swap out the calls to OR-Tools with API calls to the rideOS platform’s Fleet Planner API instead. In this way, the experimental setup is identical, other than the optimization solver itself, thus allowing an apples-to-apples comparison.

 

Methodology

Dataset:

The NYC Taxi and Limousine Commission (TLC) provides publicly available data on their historical trip information. In the dataset, each trip defined the following:

  • Origin zone id
  • Destination zone id
  • Number of passengers
  • Trip pickup time
  • Trip dropoff time

Due to privacy reasons, the exact latitude and longitude of the origins and destinations are not provided, and are instead converted into zones. Below is an image showing the zones in Manhattan:

TLC Taxi Zones in Manhattan. Source: City of New York

 

The bounding polygons of these zones are provided on the TLC trip record website as shape files and as geojson files, which we used in our experiments.

The TLC dataset contains trip information from 2009 to 2020, and for the purposes of this experiment, we randomly selected 100 trips from January 1st 2019, 12:00:00 Eastern till 12:59:59 Eastern, that had origins and destinations in the Manhattan zones.

 

Defining the Osiris simulation scenario

Since the exact pickup and dropoff locations are unknown, Osiris automatically generates a scenario (with specific pickup and dropoff latitude-longitudes, vehicle positions etc) from the Manhattan zones as defined above. For the purposes of this experiment, we generated a single simulation scenario, so that both the rideOS platform and OR-Tools run the exact same scenario. Again, this ensures that a high fidelity apples-to-apples comparison of rideOS optimization against OR-Tools.

 

Simulation Setup

The simulation is set up to run the scenario as realistically as possible, in order to accurately measure the performance of the rideOS platform against OR-Tools. The simulation ran in the following way:

  • The simulator kept an internal clock that started at 2019-01:01 12:00.
  • As the simulated clock progressed, trips would be requested at their appropriate time.
    • For example, a trip request in the dataset at 2019-01-01 12:13 would only be requested when the simulated clock reached 2019-01-01 12:13.
  • Simulated vehicles follow real-time traffic and routes provided by the rideOS platform.
    • For example, if a trip is requested at 2019-01-01 12:13, there may be vehicles en-route on their current plans with prior trips, and the vehicles’ plans and routes will be updated accordingly based on the new trip assignment.

As the simulation is being run, logs are written of the states of the vehicles and trips, and metrics are computed at the end of the simulation.

 

Metrics Computed

We considered five metrics:

  • # successful trips: How many trips were successful, e.g., a vehicle was assigned, the passenger was picked up and then dropped off.
  • Utilization: What % of time is a vehicle active, i.e., is it actively carrying at least 1 passenger?
  • Capacity utilization: How many passengers are in a vehicle?
  • Wait time: How long did a passenger have to wait before being picked up?
  • Active time: How long was a passenger on board the vehicle between pickup and dropoff?

 

For the latter 4 metrics, we computed the minimum, 25th percentile, median, 75th percentile, and maximum values. Below are box and whisker charts of these metric values. The upper and lower ends of the boxes show the 75th and 25th percentiles, and the whiskers show the maximum and minimum.

 

Results & Analysis

100% of the trips were successful, i.e., the passenger(s) were picked up and dropped off, both when the rideOS platform and OR-Tools ran the simulation scenario.

 

Fleet Utilization

The mean fleet utilization is similar for both rideOS and OR-Tools, although OR-Tools has a larger spread. In particular, the median fleet utilization (not shown in the figure above) is higher with the rideOS platform (43.8%) than compared to OR-Tools (37.7%), this showing that the rideOS platform makes better use of the fleet.

 

Capacity Utilization

The capacity utilization is comparable for both rideOS and OR-Tools, although OR-Tools again has a larger spread.

 

Wait time

The wait time on the rideOS platform is significantly lower than that of OR-Tools, showing that the rideOS platform leads to a better customer experience.

100% of the trips were successful, i.e., the passenger(s) were picked up and dropped off, both when the rideOS platform and OR-Tools ran the simulation scenario.

 

Active time

Similarly, the active time is lower with the rideOS platform compared to OR-Tools, again showing that the rideOS platform leads to a better customer experience.

 

In conclusion, the rideOS platform outperforms OR-Tools in terms of wait time, active time, and fleet utilization, and is comparable in terms of capacity utilization.

 

Running your fleet on the rideOS platform will result in substantially better performance in terms of fleet efficiency and customer satisfaction. Contact us for more info.