SitePoint PHPCase Study: Optimizing CommonMark Markdown Parser with Blackfire.io (23.11.2017, 06:12 UTC)

As you may know, I am the author and maintainer of the PHP League's CommonMark Markdown parser. This project has three primary goals:

  1. fully support the entire CommonMark spec
  2. match the behavior of the JS reference implementation
  3. be well-written and super-extensible so that others can add their own functionality.

This last goal is perhaps the most challenging, especially from a performance perspective. Other popular Markdown parsers are built using single classes with massive regex functions. As you can see from this benchmark, it makes them lightning fast:

Library Avg. Parse Time File/Class Count
Parsedown 1.6.0 2 ms 1
PHP Markdown 1.5.0 4 ms 4
PHP Markdown Extra 1.5.0 7 ms 6
CommonMark 0.12.0 46 ms 117

Unfortunately, because of the tightly-coupled design and overall architecture, it's difficult (if not impossible) to extend these parsers with custom logic.

For the League's CommonMark parser, we chose to prioritize extensibility over performance. This led to a decoupled object-oriented design which users can easily customize. This has enabled others to build their own integrations, extensions, and other custom projects.

The library's performance is still decent --- the end user probably can't differentiate between 42ms and 2ms (you should be caching your rendered Markdown anyway). Nevertheless, we still wanted to optimize our parser as much as possible without compromising our primary goals. This blog post explains how we used Blackfire to do just that.

Profiling with Blackfire

Blackfire is a fantastic tool from the folks at SensioLabs. You simply attach it to any web or CLI request and get this awesome, easy-to-digest performance trace of your application's request. In this post, we'll be examining how Blackfire was used to identify and optimize two performance issues found in version 0.6.1 of the league/commonmark library.

Let's start by profiling the time it takes league/commonmark to parse the contents of the CommonMark spec document:

Initial benchmark of league/commonark 0.6.1

Later on we'll compare this benchmark to our changes in order to measure the performance improvements.

Quick side-note: Blackfire adds overhead while profiling things, so the execution times will always be much higher than usual. Focus on the relative percentage changes instead of the absolute "wall clock" times.

Optimization 1

Looking at our initial benchmark, you can easily see that inline parsing with InlineParserEngine::parse() accounts for a whopping 43.75% of the execution time. Clicking this method reveals more information about why this happens:

Detailed view of InlineParseEngine::parse()

Here we see that InlineParserEngine::parse() is calling Cursor::getCharacter() 79,194 times --- once for every single character in the Markdown text. Here's a partial (slightly-modified) excerpt of this method from 0.6.1:

public function parse(ContextInterface $context, Cursor $cursor)
{
    // Iterate through every single character in the current line
    while (($character = $cursor->getCharacter()) !== null) {
        // Check to see whether this character is a special Markdown character
        // If so, let it try to parse this part of the string
        foreach ($matchingParsers as $parser) {
            if ($res = $parser->parse($context, $inlineParserContext)) {
                continue 2;
            }
   

Truncated by Planet PHP, read more at the original (another 2728 bytes)

Link
Fabien PotencierSymfony Flex Private Repositories (23.11.2017, 00:00 UTC)

Many Flex early adopters asked for it. The Symfony Flex server now supports private recipes repositories as announced during my keynote at SymfonyCon Cluj.

Creating a repository for your private recipes is easy. Create a regular Github repository (probably a private one) to store the recipes. The directory structure is the same as for the official Flex recipes repositories. Then, register the repository as a recipes repository. Done.

After registration, the repository will behave in the exact same way as the official ones, with the same features like automatic validation of pull requests by the Flex bot, and a dedicated endpoint to test a pull request on a project.

As for the main recipes repository, but unlike the contrib one, private repositories can define aliases and override existing ones. Your aliases take precedence over the official ones. If you don't like our default choice for the admin or orm aliases, override them. You can also override a recipe to customize the package's default configuration.

There are some differences with the public repositories though: of course, you can define recipes for your private packages. Auto-merge of pull requests is not supported yet. And for Makefile lovers, I have some good news: private repositories support Makefile (as you have full control of your stack).

How do you configure the projects that can have access to your private repository? In a config.json file located at the root directory of the repository. Under the projects key, list the project's ULID (as found in composer.json or via composer config extra.symfony.id):

{
    "projects": {
        "01BWTQ6H4KJFHRZ240CXBN6S6A": "my super secret project",
        "01BYV69F2S7ECNN3N76K82BV4W": "demo"
    }
}

While in beta, private repositories are free.

Happy private Flex!

Link
SitePoint PHPHow to Optimize Docker-based CI Runners with Shared Package Caches (21.11.2017, 17:00 UTC)

At Unleashed Technologies we use Gitlab CI with Docker runners for our continuous integration testing. We've put significant effort into speeding up the build execution speeds. One of the optimizations we made was to share a cache volume across all the CI jobs, allowing them to share files like package download caches.

Continue reading %How to Optimize Docker-based CI Runners with Shared Package Caches%

Link
Voices of the ElePHPantInterview with Larry Amond and Mike Pavlak (21.11.2017, 12:30 UTC) Link
Fabien PotencierSymfony 4: An Update on Flex (21.11.2017, 00:00 UTC)

Symfony 4 is just around the corner. And Symfony Flex is one of the main selling points for the upgrade. Developers love the new philosophy. And a lot of changes happened since my last blog post. Let me recap the recent changes that you might not be aware of. Most of these changes were prompted by feedback from early adopters.

Using recipes from the contrib repository has became easier. The first time you require a package for which a contrib recipe exists, Flex now asks you the permission to execute the recipe. It also lets you switch the "accept contrib recipes" flag for the project (so it does not ask the question again). Much better experience, and better discoverability.

One issue people have had a lot is recipes installed multiple-times. I thought Flex got you covered except under some rare circumstances, but apparently those rare conditions are a bit more frequent than I anticipated. So, instead of trying to make clever hooks into the Composer life-cycle to determine if a recipe should be applied or not, each project now has a symfony.lock file where Flex's state is stored (this file must be committed in the project's code repository). Problem solved. And that lock file opens up some nice opportunities... but that's for another iteration.

Makefile support proves to be a nightmare. Despite several patches to the recipes having some Makefile support (support for Windows and whatnot), it was a never ending battle. At the end of the day, we realize that people had wrong expectations about the Makefile, and that make is not so standard. Instead of fighting the tool, we decided to add symfony/console as a requirement. Done. Simple. Efficient. And yes, Makefile support was introduced just as a facade to the optional Symfony Console component. You can still use a Makefile for your project of course. And I still think that this is a good idea.

The minimum version supported by Flex is now PHP 7.0 instead of PHP 7.1 previously to let more people use Flex. It's especially important for projects using Symfony 3.4, which is a long term support release. That will make the upgrade path to Symfony 4 smoother as well.

Having beta testers helped find bugs you would never have expected. When upgrading Flex, people had weird issues. That was because of how Composer plugins work. Some Flex classes are loaded early on by Composer. And if down the road, Flex is updated, you can have a situtation where a class from the newest version is mixed with classes from the old version. Not a good situation to be in. Now, Flex loads all its classes preemptively to be sure it is always in a consistent state.

In terms of features, Flex is now able to auto-register almost any bundle for which there are no recipes. And we added support for environment variables in PHPUnit configuration. Using Flex is also much faster as there are less roundtrips with the server (only one request per Composer session most of the time).

Regarding documentation, we have also started to document the new workflow with updates to the upgrade tutorial and the best practices document. They now take into account changes introduced by Flex. However, keep in mind that documentation updates are still on-going.

Last, but not least, the Symfony Flex Server has a new UI, which allows you to better discover available recipes, aliases, and packs.

Happy Flex!

Link
SitePoint PHPHow to Optimize SQL Queries for Faster Sites (20.11.2017, 17:00 UTC)

This article was originally published on the Delicious Brains blog, and is republished here with permission.

You know that a fast site == happier users, improved ranking from Google, and increased conversions. Maybe you even think your WordPress site is as fast as it can be: you've looked at site performance, from the best practices of setting up a server, to troubleshooting slow code, and offloading your images to a CDN, but is that everything?

With dynamic, database-driven websites like WordPress, you might still have one problem on your hands: database queries slowing down your site.

In this post, I’ll take you through how to identify the queries causing bottlenecks, how to understand the problems with them, along with quick fixes and other approaches to speed things up. I’ll be using an actual query we recently tackled that was slowing things down on the customer portal of deliciousbrains.com.

Identification

The first step in fixing slow SQL queries is to find them. Ashley has sung the praises of the debugging plugin Query Monitor on the blog before, and it’s the database queries feature of the plugin that really makes it an invaluable tool for identifying slow SQL queries. The plugin reports on all the database queries executed during the page request. It allows you to filter them by the code or component (the plugin, theme or WordPress core) calling them, and highlights duplicate and slow queries:

Query Monitor results

If you don’t want to install a debugging plugin on a production site (maybe you’re worried about adding some performance overhead) you can opt to turn on the MySQL Slow Query Log, which logs all queries that take a certain amount of time to execute. This is relatively simple to configure and set up where to log the queries to. As this is a server-level tweak, the performance hit will be less that a debugging plugin on the site, but should be turned off when not using it.

Understanding

Once you have found an expensive query that you want to improve, the next step is to try to understand what is making the query slow. Recently during development to our site, we found a query that was taking around 8 seconds to execute!

SELECT
    l.key_id,
    l.order_id,
    l.activation_email,
    l.licence_key,
    l.software_product_id,
    l.software_version,
    l.activations_limit,
    l.created,
    l.renewal_type,
    l.renewal_id,
    l.exempt_domain,
    s.next_payment_date,
    s.status,
    pm2.post_id AS 'product_id',
    pm.meta_value AS 'user_id'
FROM
    oiz6q8a_woocommerce_software_licences l
        INNER JOIN
    oiz6q8a_woocommerce_software_subscriptions s ON s.key_id = l.key_id
        INNER JOIN
    oiz6q8a_posts p ON p.ID = l.order_id
        INNER JOIN
    oiz6q8a_postmeta pm ON pm.post_id = p.ID
        AND pm.meta_key = '_customer_user'
        INNER JOIN
    oiz6q8a_postmeta pm2 ON pm2.meta_key = '_software_product_id'
        AND pm2.meta_value = l.software_product_id
WHERE
    p.post_type = 'shop_order'
        AND pm.meta_value = 279
ORDER BY s.next_payment_date

We use WooCommerce and a customized version of the WooCommerce Software Subscriptions plugin to run our plugins store. The purpose of this query is to get all subscriptions for a customer where we know their customer number. WooCommerce has a somewhat complex data model, in that even though an order is stored as a custom post type, the id of the customer (for stores where each customer gets a WordPress user created for them) is not stored as the post_author, but instead as a piece of post meta data. There are also a couple of joins to custom tables created by the software subscriptions plugin. Let’s dive in to understand the query more.

MySQL is your Friend

MySQL has a handy statement DESCRIBE which can be used to out

Truncated by Planet PHP, read more at the original (another 4509 bytes)

Link
Piotr PasichHow To Setup Own Bitcoin Simulation Network (20.11.2017, 07:00 UTC)
Screen-Shot-2017-10-23-at-5.59.30-PM-e1508796050361Paying for goods with Bitcoin is becoming more and more popular. If you run, maintain or develop an e-commerce website, ICO project, exchange, or trading system, you may be interested in a quick way of integrating with the Bitcoin blockchain. This article will show one of many ways of installing a Bitcoin mining software and […]
Link
Stefan KoopmanschapSilex is (almost) dead, long live my-lex (17.11.2017, 10:30 UTC)

SymfonyCon is happening in Cluj and on Thursday the keynote by Fabien Potencier announced some important changes. One of the most important announcements was the EOL of Silex in 2018.

EOL next year for Silex! #SymfonyCon ( -@gbtekkie)

Silex

Silex has been and is still an important player in the PHP ecosystem. It has played an extremely important role in the Symfony ecosystem as it showed many Symfony developers that there was more than just the full Symfony stack. It was also one of the first microframeworks that showed the PHP community the power of working with individual components, and how you can glue those together to make an extremely powerful foundation to build upon which includes most of the best practices.

Why EOL?

Now I wasn't at the keynote so I can only guess to the reasons, but it does make sense to me. When Silex was released the whole concept of taking individual components to build a microframework was pretty new to PHP developers. The PHP component ecosystem was a lot more limited as well. A huge group of PHP developers was used to working with full stack frameworks, so building your own framework (even with components) was by many deemed to be reinventing the wheel.

Fastforward to 2017 and a lot of PHP developers are by now used to individual components. Silex has little to prove on that topic. And with Composer being a stable, proven tool, the PHP component ecosystem growing every day and now the introduction of Symfony Flex to easily setup and manage projects maintaining a seperate microframework based on Symfony components is just an overhead. Using either Composer or Symfony Flex, you can set up a project similar to an empty Silex project in a matter of minutes.

Constructicons

I have been a happy user of Composer with individual components for a while now. One of my first projects with individual components even turned into a conference talk. I'll update the talk soon, as I have since found a slightly better structure, and if I can make the time for it, I'll also write something about this new and improved structure. I've used it for a couple of projects now and I'm quite happy with this structure. I also still have to play with Symfony Flex. It looks really promising and I can't wait to give it a try.

So the "my-lex" in the title, what is that about? It is about the choice you now have. You can basically build your own Silex using either Composer and components or Symfony Flex. I would've laughed hard a couple of years ago if you'd said to me that I would say this but: Build your own framework!

Is Silex being EOL'ed a bad thing?

No. While it is sad to see such an important project go I think by now the Symfony and PHP ecosystems have already gone past the point of needing Silex. Does this mean we don't need microframeworks anymore? I won't say that, but with Slim still going strong the loss of Silex isn't all that bad. And with Composer, Flex and the huge amount of PHP components, you can always build a microframework that suits your specific needs.

The only situation where Silex stopping is an issue is for open source projects such as Bolt (who already anticipated this that are based on Silex, as well as of course your personal or business projects based on Silex. While this software will keep on working, you won't get new updates of the core of those projects, so eventually you'll have to put in effort to rewrite it to something else.

Link
Remi ColletFedora 27: changes in httpd and php (17.11.2017, 08:42 UTC)

The Apache HTTP server and PHP configuration have changed in Fedora 27, here is some explanations.

1. Switch of the Apache HTTP server in event mode

Since the first days of the distribution, the severs use the prefork MPM.

For obvious performance reasons, we choose to follow the upstream project recommandations and to use the event MPM by default.

This change is also required to have the full benefit and feature of the HTTP/2 protocol via mod_http2.

2. The problem of mod_php

The mod_php module is only supported when the prefork MPM is used

In the PHP documentation, we can read:

Warning We do not recommend using a threaded MPM in production with Apache 2.

And, indeed, we already have some bug reports about crashes in this configuration.

So it doesn't make sense to keep mod_php by default.

Furthermore, this module have some annoying limitations:

  • integrated in the web server, it shares its memory, which may have some negative security impacts
  • a single version can be loaded

3. Using FastCGI

For many years, we are working to make the PHP execution as much flexible as possible, using various combinations, without configuration change:

  • httpd + mod_php
  • httpd + php-fpm (when mod_php is disabled or missing and with a running php-fpm server)
  • nginx + php-fpm

The FPM way have become the default recommend configuration for a safe PHP execution:

  • support of multiple web servers (httpd, nginx, lighttpd)
  • frontend isolation for security
  • multiple backends
  • micro-services architecture
  • containers (docker)
  • multiple versions of PHP

4. FPM by default

Since Fedora 27, mod_php ZTS (multi-threaded) is still provided, but disabled, so FastCGI is now used by default.

To not break existing configuration during the distribution upgrade, and to have a working server after installation, we choose to implement some solutions, probably temporarily:

  • the php package have a optional dependency on the php-fpm package, so it is now installed by default
  • the httpd service have a dependency on the php-fpm service, so it is started automatically

5. Known issues

5.1. Configuration change

After a configuration change, or after a new extension installation, it is now required to restart the php-fpm service.

5.2. Configuration files

With mod_php, it is common to to use the php_value or php_flag directives in the Apache HTTP server configuration or in some .htaccess file.

It is now required to use the php_value or php_flag directives in the FPM pool configuration file, or to use some .user.ini file in the application directory.

5.3 Users

By default httpd and php-fpm run using the apache account. If you need to change it for httpd, you also have to change the default pool configuration , in /etc/php-fpm.d/www.conf

user = foo
listen.acl_users = foo

6. Switching back to mod_php

If you really want to keep using (temporarily) mod_php, this is still possible, either way:

  • Switch back to prefork MPM in the /etc/httpd/conf.modules.d/00-mpm.conf file
 LoadModule mpm_prefork_module modules/mod_mpm_prefork.so
 #LoadModule mpm_worker_module modules/mod_mpm_worker.so
 #LoadModule mpm_event_module modules/mod_mpm_event.so
  • Enable the module in the /etc/httpd/conf.modules.d/15-php.conf file. Warning, this configuration will not be supported, no bug report will be accepted.
 # ZTS module is not supported, so FPM is preferred
 LoadModule php7_modu

Truncated by Planet PHP, read more at the original (another 606 bytes)

Link
SitePoint PHPHow to Read Big Files with PHP (Without Killing Your Server) (16.11.2017, 18:00 UTC)

It’s not often that we, as PHP developers, need to worry about memory management. The PHP engine does a stellar job of cleaning up after us, and the web server model of short-lived execution contexts means even the sloppiest code has no long-lasting effects.

There are rare times when we may need to step outside of this comfortable boundary --- like when we're trying to run Composer for a large project on the smallest VPS we can create, or when we need to read large files on an equally small server.

Fragmented terrain

It’s the latter problem we'll look at in this tutorial.

The code for this tutorial can be found on GitHub.

Measuring Success

The only way to be sure we’re making any improvement to our code is to measure a bad situation and then compare that measurement to another after we’ve applied our fix. In other words, unless we know how much a “solution” helps us (if at all), we can’t know if it really is a solution or not.

There are two metrics we can care about. The first is CPU usage. How fast or slow is the process we want to work on? The second is memory usage. How much memory does the script take to execute? These are often inversely proportional --- meaning that we can offload memory usage at the cost of CPU usage, and vice versa.

In an asynchronous execution model (like with multi-process or multi-threaded PHP applications), both CPU and memory usage are important considerations. In traditional PHP architecture, these generally become a problem when either one reaches the limits of the server.

It's impractical to measure CPU usage inside PHP. If that’s the area you want to focus on, consider using something like top, on Ubuntu or macOS. For Windows, consider using the Linux Subsystem, so you can use top in Ubuntu.

For the purposes of this tutorial, we’re going to measure memory usage. We’ll look at how much memory is used in “traditional” scripts. We’ll implement a couple of optimization strategies and measure those too. In the end, I want you to be able to make an educated choice.

The methods we’ll use to see how much memory is used are:

// formatBytes is taken from the php.net documentation

memory_get_peak_usage();

function formatBytes($bytes, $precision = 2) {
    $units = array("b", "kb", "mb", "gb", "tb");

    $bytes = max($bytes, 0);
    $pow = floor(($bytes ? log($bytes) : 0) / log(1024));
    $pow = min($pow, count($units) - 1);

    $bytes /= (1 << (10 * $pow));

    return round($bytes, $precision) . " " . $units[$pow];
}

We’ll use these functions at the end of our scripts, so we can see which script uses the most memory at one time.

What Are Our Options?

There are many approaches we could take to read files efficiently. But there are also two likely scenarios in which we could use them. We could want to read and process data all at the same time, outputting the processed data or performing other actions based on what we read. We could also want to transform a stream of data without ever really needing access to the data.

Let’s imagine, for the first scenario, that we want to be able to read a file and create separate queued processing jobs every 10,000 lines. We’d need to keep at least 10,000 lines in memory, and pass them along to the queued job manager (whatever form that may take).

For the second scenario, let’s imagine we want to compress the contents of a particularly large API response. We don’t care what it says, but we need to make sure it’s backed up in a compressed form.

In both scenarios, we need to read large files. In the first, we need to know what the data is. In the second, we don’t care what the data is. Let’s explore these options…

Reading Files, Line By Line

There are many functions for working with files. Let’s combine a few into a naive file reader:

// from memory.php

function formatBytes($bytes, $precision = 2) {
    $units = array("b", "kb", "mb", "gb", "tb");

    $bytes = max($bytes, 0);
    $pow = floor(($bytes ? log($bytes) : 0) / log(1024));
    $pow = min($pow, count($units) - 1);

    $bytes /= (1 << (10 * $pow));

    return round($bytes, $precision) . " " . $units[$pow];
}

print formatBytes(memory_get_peak_usage());

// from reading-files-line-by-line-1.php

function readTheFile($path) {
    $lines = [];
    $handle = fopen($path, "r");

    while(!feof($ha

Truncated by Planet PHP, read more at the original (another 7383 bytes)

Link
LinksRSS 0.92   RDF 1.
Atom Feed   100% Popoon
PHP5 powered   PEAR
ButtonsPlanet PHP   Planet PHP
Planet PHP