Laravel pipelines are a powerful feature that allows you to build complex workflows by breaking down operations into smaller, reusable stages. Pipelines offer a more efficient, readable, and scalable approach to handling data transformations in your Laravel applications. In this post, we'll explore the basics of pipelines, their benefits, and how to use them to build APIs.
Laravel pipelines are a design pattern that allows you to create a sequence of stages, where each stage receives input data, performs some operation, and then passes the output to the next stage. This pattern is especially useful for processing large data sets, where you need to perform multiple operations on the data before returning the final result.
By using pipelines, you can:
First, let's modify our users migration (this file is likely named 2014_10_12_000000_create_users_table.php in your repository). This modification will introduce the 'dob' field which we will later reference in our API.
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
return new class extends Migration
{
/**
* Run the migrations.
*/
public function up(): void
{
Schema::create('users', function (Blueprint $table) {
$table->id();
$table->string('name');
$table->string('email')->unique();
$table->timestamp('email_verified_at')->nullable();
$table->string('password');
$table->string('timezone')->index();
$table->date('dob')->index();
$table->rememberToken();
$table->timestamps();
});
}
/**
* Reverse the migrations.
*/
public function down(): void
{
Schema::dropIfExists('users');
}
};
Now we need to migrate the table to the database:
php artisan migrateFrom here we will update our UserFactory.php to include the `dob` field which we can generate a fake date for on demand:
namespace Database\Factories;
use Illuminate\Database\Eloquent\Factories\Factory;
use Illuminate\Support\Str;
class UserFactory extends Factory
{
public function definition(): array
{
return [
'name' => fake()->name(),
'email' => fake()->unique()->safeEmail(),
'email_verified_at' => now(),
'timezone' => fake()->timezone,
'password' => '$2y$10$92IXUNpkjO0rOQ5byMi.Ye4oKoEa3Ro9llC/.og/at2.uheWG/igi', // password
'dob' => $this->faker->date,
'remember_token' => Str::random(10),
];
}
}
Now we need to get data into your database. Let's fire up Tinker:
php artisan tinkerThis will now place us in a tinker session. Now type the following into the terminal, then press ENTER. This will generate 1000 user records into your database:
\App\Models\User::factory()->times(1000)->create();Now that we have a decently sized data set, we'll close Tinker and create a controller:
php artisan make:controller UserSearchController -iLet's add the route in our api.php:
Route::get('users/search', \App\Http\Controllers\UserSearchController::class);We'll now take that controller and bootstrap the pipeline:
namespace App\Http\Controllers;
use App\Models\User;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Pipeline;
class UserSearchController extends Controller
{
/**
* Handle the incoming request.
*
* @param Request $request
*/
public function __invoke(Request $request)
{
$pipelines = [];
return Pipeline::send(User::query())
->through($pipelines)
->thenReturn()
->paginate();
}
}
Right now the code would return the same as if we did this:
User::paginate()Before we start adding stages to our pipeline lets visit our endpoint in the browser:
http://yourdomain.test/api/users/search{
"current_page": 1,
"data": [
{
"id": 1,
"name": "Clara Runte",
"email": "jmurazik@example.org",
"email_verified_at": "2023-03-16T20:50:38.000000Z",
"timezone": "America/Sitka",
"dob": "1995-12-14",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 2,
"name": "Vicky Brown DVM",
"email": "kohler.mikel@example.net",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Africa/Djibouti",
"dob": "1976-06-16",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 3,
"name": "Charlotte Gutkowski",
"email": "clifford.schumm@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Africa/Gaborone",
"dob": "1971-09-23",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 4,
"name": "Mariana Hamill",
"email": "lbrakus@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Europe/Ljubljana",
"dob": "1989-05-13",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 5,
"name": "Prof. Ima Zemlak",
"email": "heaven.konopelski@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "America/Rio_Branco",
"dob": "1985-09-04",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 6,
"name": "Anissa Wiegand",
"email": "turcotte.bernhard@example.com",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "America/Yakutat",
"dob": "1978-10-25",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 7,
"name": "Vanessa Metz",
"email": "carson09@example.net",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Europe/Oslo",
"dob": "1987-07-26",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 8,
"name": "Dr. Alexandre Kunze Jr.",
"email": "caden.beahan@example.net",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "America/Eirunepe",
"dob": "2017-05-19",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 9,
"name": "Darrel Barton",
"email": "casimer64@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "America/Matamoros",
"dob": "1973-11-02",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 10,
"name": "Prof. Darrick Kozey",
"email": "rosalee56@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Africa/Juba",
"dob": "2020-12-15",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 11,
"name": "Gage Crooks",
"email": "devante.erdman@example.net",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "America/Costa_Rica",
"dob": "1985-06-01",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 12,
"name": "Candelario Kub",
"email": "austin.hartmann@example.net",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "America/Punta_Arenas",
"dob": "2006-02-10",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 13,
"name": "Sammy Kirlin",
"email": "tomasa87@example.com",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Africa/Nouakchott",
"dob": "1972-02-25",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 14,
"name": "Lysanne Walter",
"email": "katheryn.tromp@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Asia/Urumqi",
"dob": "2004-06-27",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 15,
"name": "Kurtis Renner",
"email": "xweber@example.net",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "America/Araguaina",
"dob": "1998-01-13",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
}
],
"first_page_url": "http://yourdomain.test/users/search?page=1",
"from": 1,
"last_page": 67,
"last_page_url": "http://yourdomain.test/users/search?page=67",
"links": [
{
"url": null,
"label": "« Previous",
"active": false
},
{
"url": "http://yourdomain.test/users/search?page=1",
"label": "1",
"active": true
},
{
"url": "http://yourdomain.test/users/search?page=2",
"label": "2",
"active": false
},
{
"url": "http://yourdomain.test/users/search?page=3",
"label": "3",
"active": false
},
{
"url": "http://yourdomain.test/users/search?page=4",
"label": "4",
"active": false
},
{
"url": "http://yourdomain.test/users/search?page=5",
"label": "5",
"active": false
},
{
"url": "http://yourdomain.test/users/search?page=6",
"label": "6",
"active": false
},
{
"url": "http://yourdomain.test/users/search?page=7",
"label": "7",
"active": false
},
{
"url": "http://yourdomain.test/users/search?page=8",
"label": "8",
"active": false
},
{
"url": "http://yourdomain.test/users/search?page=9",
"label": "9",
"active": false
},
{
"url": "http://yourdomain.test/users/search?page=10",
"label": "10",
"active": false
},
{
"url": null,
"label": "...",
"active": false
},
{
"url": "http://yourdomain.test/users/search?page=66",
"label": "66",
"active": false
},
{
"url": "http://yourdomain.test/users/search?page=67",
"label": "67",
"active": false
},
{
"url": "http://yourdomain.test/users/search?page=2",
"label": "Next »",
"active": false
}
],
"next_page_url": "http://yourdomain.test/users/search?page=2",
"path": "http://yourdomain.test/users/search",
"per_page": 15,
"prev_page_url": null,
"to": 15,
"total": 1000
}
Since UserFactory creates data at random, your data points' will not look exactly like mine.
Now we'll create a class called 'ByEmail' which will serve as our filter allowing us to "filter by email."
namespace App\Filters;
use Closure;
use Illuminate\Database\Eloquent\Builder;
use Illuminate\Http\Request;
class ByEmail
{
public function __construct(public Request $request){}
public function handle(Builder $query, Closure $next)
{
return $next($query)
->when($this->request->has('email'),
fn($query) => $query->where('email', 'regexp', $this->request->email)
);
}
}
So what exactly is going on here?
public function __construct(public Request $request){}We're asking the service container to return us the current request coming into our application and assign it to a public attribute called $request which we can reference anytime by using $this->request.
Now lets examine the handle() method.
public function handle(Builder $query, Closure $next)
{
return $next($query)
->when($this->request->has('email'),
fn($query) => $query->where('email', 'regexp', $this->request->email)
);
}The first parameter, Builder $query, serves as the payload being sent into our pipeline, which in our case is the Eloquent Query Builder. The second parameter, Closure $next, is what is used to send our payload, along with any modifications made to it, onto the next stage in the pipeline.
The when() method is telling the query builder that "when" there is a email being provided in our request we want to apply the query fn($query) => $query->where('email', 'regexp', $this->request->email) which will search for any email addresses that contain any character provided by the "email" portion of the request.
We now need to add this filter to our pipeline. We'll do this by adding the fully qualified name to the $pipelines array variable in our controller:
namespace App\Http\Controllers;
use App\Models\User;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Pipeline;
class UserSearchController extends Controller
{
/**
* Handle the incoming request.
*
* @param Request $request
*/
public function __invoke(Request $request)
{
$pipelines = [
\App\Filters\ByEmail::class
];
return Pipeline::send(User::query())
->through($pipelines)
->thenReturn()
->paginate();
}
}With our pipeline being aware of our filter, let's test this by visiting our API URL from earlier, but we'll be sure to add the email reference to our query string:
http://yourdomain.test/api/users/search?email=.orgIn my example I'm telling our endpoint we want to filter by email addresses which contain .org. As you'll see our results set now includes only records with email addresses containing '.org' addresses:
{
"current_page": 1,
"data": [
{
"id": 1,
"name": "Clara Runte",
"email": "jmurazik@example.org",
"email_verified_at": "2023-03-16T20:50:38.000000Z",
"timezone": "America/Sitka",
"dob": "1995-12-14",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 3,
"name": "Charlotte Gutkowski",
"email": "clifford.schumm@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Africa/Gaborone",
"dob": "1971-09-23",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 4,
"name": "Mariana Hamill",
"email": "lbrakus@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Europe/Ljubljana",
"dob": "1989-05-13",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 5,
"name": "Prof. Ima Zemlak",
"email": "heaven.konopelski@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "America/Rio_Branco",
"dob": "1985-09-04",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 9,
"name": "Darrel Barton",
"email": "casimer64@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "America/Matamoros",
"dob": "1973-11-02",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 10,
"name": "Prof. Darrick Kozey",
"email": "rosalee56@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Africa/Juba",
"dob": "2020-12-15",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 14,
"name": "Lysanne Walter",
"email": "katheryn.tromp@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Asia/Urumqi",
"dob": "2004-06-27",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 22,
"name": "Zula Hammes",
"email": "hill.rosie@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Pacific/Midway",
"dob": "2008-10-08",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 23,
"name": "Brandon Reichert",
"email": "aliya.rohan@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Africa/Dakar",
"dob": "2022-01-14",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 27,
"name": "Michele Labadie",
"email": "issac84@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "America/Blanc-Sablon",
"dob": "2003-08-25",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 28,
"name": "Dr. Erik Metz IV",
"email": "jveum@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Africa/Luanda",
"dob": "1971-05-16",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 30,
"name": "Macy Beier",
"email": "nicolas.alexandrea@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Pacific/Pago_Pago",
"dob": "1992-03-31",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 33,
"name": "Mrs. Donna Schuppe DDS",
"email": "vblanda@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Atlantic/Canary",
"dob": "2018-08-11",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 40,
"name": "Rebeka Hagenes V",
"email": "torphy.paula@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Asia/Srednekolymsk",
"dob": "1992-08-16",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 42,
"name": "Rey Zieme",
"email": "ilehner@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Asia/Tehran",
"dob": "1981-05-31",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
}
],
"first_page_url": "http://teachinglaravel.test/users/search?page=1",
"from": 1,
"last_page": 24,
"last_page_url": "http://teachinglaravel.test/users/search?page=24",
"links": [
{
"url": null,
"label": "« Previous",
"active": false
},
{
"url": "http://teachinglaravel.test/users/search?page=1",
"label": "1",
"active": true
},
{
"url": "http://teachinglaravel.test/users/search?page=2",
"label": "2",
"active": false
},
{
"url": "http://teachinglaravel.test/users/search?page=3",
"label": "3",
"active": false
},
{
"url": "http://teachinglaravel.test/users/search?page=4",
"label": "4",
"active": false
},
{
"url": "http://teachinglaravel.test/users/search?page=5",
"label": "5",
"active": false
},
{
"url": "http://teachinglaravel.test/users/search?page=6",
"label": "6",
"active": false
},
{
"url": "http://teachinglaravel.test/users/search?page=7",
"label": "7",
"active": false
},
{
"url": "http://teachinglaravel.test/users/search?page=8",
"label": "8",
"active": false
},
{
"url": "http://teachinglaravel.test/users/search?page=9",
"label": "9",
"active": false
},
{
"url": "http://teachinglaravel.test/users/search?page=10",
"label": "10",
"active": false
},
{
"url": null,
"label": "...",
"active": false
},
{
"url": "http://teachinglaravel.test/users/search?page=23",
"label": "23",
"active": false
},
{
"url": "http://teachinglaravel.test/users/search?page=24",
"label": "24",
"active": false
},
{
"url": "http://teachinglaravel.test/users/search?page=2",
"label": "Next »",
"active": false
}
],
"next_page_url": "http://teachinglaravel.test/users/search?page=2",
"path": "http://teachinglaravel.test/users/search",
"per_page": 15,
"prev_page_url": null,
"to": 15,
"total": 352
}
Now that we understand what's going on we'll create 2 more filters which will allow us to search by name and dob.
namespace App\Filters;
use Closure;
use Illuminate\Database\Eloquent\Builder;
use Illuminate\Http\Request;
class ByName
{
public function __construct(public Request $request){}
public function handle(Builder $query, Closure $next)
{
return $next($query)
->when($this->request->has('name'),
fn($query) => $query->where('name', 'regexp', $this->request->name)
);
}
}
namespace App\Filters;
use Closure;
use Illuminate\Database\Eloquent\Builder;
use Illuminate\Http\Request;
class ByDob
{
public function __construct(public Request $request){}
public function handle(Builder $query, Closure $next)
{
return $next($query)
->when($this->request->has('dob'),
fn($query) => $query->where('dob', 'regexp', $this->request->dob)
);
}
}
I bet you've spotted a recurring pattern in our approach. Challenge yourself by creating a way to make the code reusable. Then share with me on Twitter what you did 😀
Let's incorporate them into our controller:
namespace App\Http\Controllers;
use App\Models\User;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Pipeline;
class UserSearchController extends Controller
{
/**
* Handle the incoming request.
*
* @param Request $request
*/
public function __invoke(Request $request)
{
$pipelines = [
\App\Filters\ByEmail::class,
\App\Filters\ByName::class,
\App\Filters\ByDob::class,
];
return Pipeline::send(User::query())
->through($pipelines)
->thenReturn()
->paginate();
}
}What's really cool now is that we can stack these filters in our request. For example, lets search for .org addresses where the name contains the letter R and is born in the year 1981.
http://yourdomain.test/api/users/search?email=.org&name=R&dob=1981{
"current_page": 1,
"data": [
{
"id": 42,
"name": "Rey Zieme",
"email": "ilehner@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Asia/Tehran",
"dob": "1981-05-31",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 84,
"name": "Clinton Hettinger",
"email": "walter.vilma@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Africa/Malabo",
"dob": "1981-03-01",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 197,
"name": "Tyrique McDermott Sr.",
"email": "cloyd.jast@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "America/Montevideo",
"dob": "1981-07-13",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 219,
"name": "Dr. Patsy Hoeger",
"email": "smitham.audra@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Europe/Bratislava",
"dob": "1981-02-03",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
},
{
"id": 877,
"name": "Mrs. Laisha Reichel MD",
"email": "leonor42@example.org",
"email_verified_at": "2023-03-16T20:50:39.000000Z",
"timezone": "Pacific/Niue",
"dob": "1981-09-20",
"created_at": "2023-03-16T20:50:39.000000Z",
"updated_at": "2023-03-16T20:50:39.000000Z"
}
],
"first_page_url": "http://teachinglaravel.test/users/search?page=1",
"from": 1,
"last_page": 1,
"last_page_url": "http://teachinglaravel.test/users/search?page=1",
"links": [
{
"url": null,
"label": "« Previous",
"active": false
},
{
"url": "http://teachinglaravel.test/users/search?page=1",
"label": "1",
"active": true
},
{
"url": null,
"label": "Next »",
"active": false
}
],
"next_page_url": null,
"path": "http://teachinglaravel.test/users/search",
"per_page": 15,
"prev_page_url": null,
"to": 5,
"total": 5
}
In conclusion, Laravel pipelines are a powerful tool that can simplify your code and streamline your workflow. By breaking down complex processes into small, manageable steps, pipelines make it easier to build scalable and maintainable applications.
Whether you're working with data transformations, API requests, or any other type of data processing, Laravel pipelines provide an elegant and efficient way to handle the flow of data through your application. By using the techniques outlined in this article, you can harness the power of pipelines to take your Laravel development to the next level.
So don't be afraid to give pipelines a try! With their flexible and customizable architecture, you can build pipelines that are tailored to your specific needs and help you build high-quality, performant applications with ease. Happy coding!