SitePoint PHPRAML, the RESTful API Modeling Language (30.1.2015, 17:00 UTC)

In a recent article I introduced Slate, a static site generator specifically designed for writing API documentation. This time around, I’m going to look at something which in many ways is even better.

But first, if you’ll indulge me for just a moment, I’d like to begin by quoting myself from that article;

[an] API is only as good as its documentation

I think it’s worth repeating, since it’s all-too-frequently overlooked, and it’s one of the motivations for this short series of articles on some of the tools out there to help you write great documentation.

Slate is a useful tool, but despite the relatively rigid structure of the pages themselves, the actual documentation is pretty freeform.

RAML (RESTful API Modeling Language) provides a structured, unambiguous format for describing a RESTful API. It allows you to describe your API; the endpoints, the HTTP methods to be used for each one, any parameters and their format, what you can expect by way of a response and more.

You can use RAML in a number of ways;

  • It can be used in the design and specification stage to describe the API you plan to build
  • You can use RAML to generate documentation
  • RAML can be used to implement interactive API consoles
  • You can test against RAML
  • It be used to generate mock API responses

In this article I’ll primarily look at generating documentation; in a later article I’ll look at how you can integrate RAML into your API testing. But first, let’s look at how to write a RAML document.

Continue reading %RAML, the RESTful API Modeling Language%

Link
Piotr PasichClassManager – You shall not pass (30.1.2015, 08:23 UTC)
NamingConventionsHi! First of all I’d like to ask you a question – what’s your name? My name is Piotr and that is derived from the Greek Πετρος (Petros) meaning “stone”.  Next to me is sitting my friend – Michael. Michael is from the Hebrew name מִיכָאֵל (Mikha’el) meaning “who is like God?” (after this article […]
Link
blog.phpdevDeveloper Security Outreach (29.1.2015, 16:00 UTC)

I’ve been thinking a lot lately about how to try to bring the security and development communities together, most specifically for PHP (see these two posts for more on that). PHP has a long standing reputation for being an insecure language that it’s had to overcome. I like to think that evidence in more recent years is helping to dissuade that, but it’s an uphill battle. Anyway, that’s not what I’m hear to talk about. True to the title of the post, I want to talk about developer outreach as it relates to security and secure development practices.

While comments were made on my two previous post about the relationship going both ways, I want to focus in on things from the perspective of the organization with the bulk of the knowledge – the application security group/company. Yes, it’s good for developers to contribute back to shared resources so both parties can benefit, but with so many new developers coming to the language every day, I see a real need for engagement. There are a lot of groups and individuals out there on the security side that specialize in training and resources to help developers write their code more securely. They provide training classes and white papers on new technologies that can be used to get the ideas across, but usually only in a limited fashion. They write blog posts about the latest exploits and vulnerabilities or even speak at conferences with case studies and their own real-world experience in the world of application security.

So, take a step back – do you see a problem with this model? Most of these things I’ve listed involve talking at the developers and not with them. Sure, some of the training classes are more hands-on and can be much more effective at getting the speaker’s ideas across. However, these kinds of resources are mostly provided if requested or actively sought out by the developer. There’s a wealth of information out there about securing applications, even PHP ones, that’s tucked away and only shown when the right Google search is performed.

Is there a solution? In thinking about it some this morning, I see a pretty obvious one – developer outreach. I’ve mentioned this same idea before in another post, but that one was more targeted towards the OWASP group and the services/resources it provides. It still surprises me when I ask in my sessions at PHP-related conference how many people have heard of OWASP and some hands go up but a lot don’t. Likewise, there’s a lot of companies out there that provide application security training (such as the Denim Group, WhiteHat Security or even SANS) but those are still presented as passive resources. Developers, by their nature, are notoriously lazy. They try to find the most efficient, most robust solutions to problems. How much would they benefit from someone from the Denim group reaching out to them or even just the PHP community as a whole and sharing what they have to offer.

Am I suggesting they hop on the various community mailing lists and start spamming them with ads for their training courses? Of course not. Here’s what I am proposing:

If you provide training, resources or any other kind of resources that developers could benefit from to create more secure applications, find an advocate (or a few) in the community of your choice and request their help to get the word out. I’d even go so far as to suggest having someone dedicated to working with communities, maybe even different people for different communities. This person should be dedicated to not only sharing what kind of things the company/group has to offer the developer community but to also act as a guide to keep them on the right path.

There’s a security subculture in just about every language out there. The key for those with the security knowledge and resources to do is to tap into it. Break into the community with a sense of humility, an open mind to learn about its members and a passion for teaching and sharing knowledge on a personal level.


Link
Matthew Weier O'PhinneyPSR-7 By Example (29.1.2015, 15:55 UTC)

PSR-7 is shaping up nicely. I pushed some updates earlier this week, and we tagged 0.6.0 of the http-message package last week for implementors and potential users to start coding against.

I'm still hearing some grumbles both of "simplify!" and "not far enough!" so I'm writing this posts to demonstrate usage of the currently published interfaces, and to illustrate both the ease of use and the completeness and robustness they offer.

First, though I want to clarify what PSR-7 is attempting.

HTTP Messages

HTTP messages are relatively simple, which is why the protocol has succeeded over the years. All messages have the following structure:


<message line>
Header: value
Another-Header: value

Message body

Headers are key/value pairs. The keys are case insensitive. Values are strings. The same header type may be emitted multiple times, in which case (typically) the values are considered as a list; in most cases, these can also be expressed by concatenating the values with comma delimiters.

The message body is a string, but typically handled by servers and clients as a stream in order to conserve memory and processing overhead. This is incredibly important when you transmit large data sets, and particularly when transmitting files. As an example, PHP natively represents the incoming request body as the stream php://input, and uses output buffers — a form of stream — to return a response.

The message line is what differentiates a request from a response.

The message line of a request is called the request line, and has the following format:


METHOD request-target HTTP/VERSION

METHOD indicates the operation requested: GET, POST, PUT, PATCH, DELETE, OPTIONS, HEAD, etc. The VERSION is typically 1.0 or 1.1 (usually 1.1 in modern web clients). The request-target is where things get complex.

A request target can be one of four different forms:

  • origin-form, which is the path and query string (if present) of the URI.
  • absolute-form, which is an absolute URI.
  • authority-form, which is the authority portion of the uri (user-info, if present; host; and port, if non-standard).
  • asterisk-form, which is the string *.

Typically, an HTTP client will use the scheme and authority from a URI to make the connection to the HTTP server, and then pass an origin-form target in the transmitted HTTP request message. However, it's perfectly valid to send the absolute URI as well. authority-form is typically only used with CONNECT requests, which are usually performed when working with a proxy server. asterisk-form is used with OPTIONS requests to get general capabilities of a web server.

In short, there's a lot of moving parts in the request-target.

Now, to make things more complicated, when we look at URIs, we have the following:


<scheme>://<authority>[/<path>][?<query string>]

The scheme, when doing HTTP requests, will be one of http or https. The path is a well-known format as well. But what about authority?

[user-info@]host[:port]

The authority always contains the host, which can be a domain name or an IP address. The port is optional, and only needs to be included if it's non-standard for the current scheme (or if the scheme is unknown). user-info is of the form:

user[:pass]

where password is optional. In fact, in current specifications, the recommendation is to never include the password in a URI, to force prompting for the value by the client.

The query string is a set of key-value pairs delimited by ampersands:

?foo=bar&baz&quz=1

Depending on the language implementation, it can also model lists and hashes:

?sort[]=ASC&sort[]=date&filter[product]=name

PHP will parse the above to:


[
    'sort' => [
        'ASC',
        'date'
    ],
    'filter' => [
        'product' => 'name'
    ],
]

So, as if the request

Truncated by Planet PHP, read more at the original (another 25222 bytes)

Link
Michael Kimsalpurpose of framework benchmarking speed (29.1.2015, 13:46 UTC)

I’ve followed the techempower benchmarks, and every now and then I check out benchmarks of various projects (usually PHP) to see what the relative state of things are. Inevitably, someone points out that “these aren’t testing anything ‘real world’ – they’re useless!”. Usually it’s from someone who’s favorite framework has ‘lost’. I used to think along the same lines; namely that “hello world” benchmarks don’t measure anything useful. I don’t hold quite the same position anymore, and I’ll explain why.

The purpose of a framework is to provide convenience, structure, guidance and hopefully some ‘best practices’ for working with the language and problem set you’re involved with. The convenience and structure come in the way of helper libraries designed to work a certain way together. In the form of code, these have a certain execution cost. What a basic “hello world” benchmark is measuring is the cost of at least some of that overhead.

What those benchmark results are telling you is “this is about the fastest this framework’s request cycle can be invoked while doing essentially nothing”. If a request cycle to do ‘hello world’ is, say, 12ms on hardware X, it will *never* be any faster than 12ms. Every single request you put through that framework will be 12ms *or slower*. Adding in cache lookups, database calls, disk access, computation, etc – those are things your application will need to do regardless of what supporting framework you’re building in (or not), but the baseline fastest performance framework X will ever achieve is 12ms.

These benchmarks are largely about establishing that baseline expectation of performance. I’d say that they’re not always necessarily presented that way, but this is largely the fault of the readers. I used to get a lot more caught up in “but framework X is ‘better'” discussions, because I was still reading them as a qualitative judgement.

But why does a baseline matter?  A standard response to slow frameworks is “they save developer time, and hardware is cheap, just get more hardware”.  Well… it’s not always that simple.  Unless you’re developing from day one to be scalable (abstracted data store instead of file system, centralized sessions vs on disk, etc), you’ll have some retooling to do.  Arguably this is a cost you’ll have to do anyway, but if you’re using a framework which has a very low baseline, you may not hit that wall for some time.  Secondly, ‘more hardware’ doesn’t really make anything go faster – it just allows you to handle more things at the same speed.  More hardware will never make anything *faster*.

“Yeah yeah yeah, but so what?”  Google uses site speed in its ranking algorithm.  What the magic formula is, no one outside Google will ever know for sure, but sites that are slower to your competitors *may* have a slight disadvantage.  Additionally, as mobile usage grows, more systems are SOA/REST based – much of your traffic will be responding to smaller calls for blobs of data.  Each request may not be huge, but they’ll need to respond quickly to give a good experience on mobile devices.  200ms response times will likely hurt you, even in the short term, as users just move to other apps, especially in the consumer space.  Business app users might be a bit more forgiving if they have to use your system for business reasons, sort of like how legions of people were stuck using IE6 for one legacy HR app.  They’ll use it, but they’ll know there are better experiences out there.

To repeat from above, throwing more hardware at the problem will never make things *faster*, so if you’ve got a slower site that needs to be measurably faster, you’ve possibly got some rearchitecting to do.  Throw some caching in, and you may get somewhat better results, but at some point, some major code modifications may be in order, and the framework that got you as far as it did may have to be abandoned for something more performant (hand rolled libraries, different language, whatever).

Of course, there’s always a maintainability aspect – I don’t recommend PHP devs throw everything away and recode their websites in C.  While this might be the most performant, it might take years to do, vs some other framework or even a different language.  I’ve incorporated Java web stacks in to my tool belt, and have some projects in Java as well as some PHP ones.  I benchmarked a simple ‘hello world’ in laravel 4

Truncated by Planet PHP, read more at the original (another 1627 bytes)

Link
SitePoint PHPIntroducing CockpitCMS – a CMS for Developers (28.1.2015, 17:00 UTC)

In this tutorial, we will see how to use Cockpit CMS to setup a CMS backend and to use the API interface provided by Cockpit to build a customized functional frontend.

Not quite like other “heavy weight”, or “fully fledged” CMS’, Cockpit is light weight and “bare”. It only provides a backend to manage meta-data and data entries, whilst the frontend is all in the developer’s hands.

Installation

A copy of Cockpit CMS can be downloaded here in zip form. After downloading, just unzip the archive to a directory on your web server that is accessible. In my case, it is mapped to: http://vagrant/cockpit.

Next, visit the installation page: http://vagrant/cockpit/install to start the installation process.

NOTE: Cockpit uses SQLite as its database engine by default. If you have not installed SQLite yet, please do so before the installation.

NOTE: Cockpit also requires that its /storage/data directory be writable. Please change the mode of that directory accordingly.

The installation is just one click. When the installation is complete, you will be greeted with this page:

Now we can log in with admin/admin and the backend administration dashboard page will be shown:

Well, we don’t have anything set up yet. Before we create our own content, let’s learn about a few key modules in Cockpit.

Modules in Cockpit

The two most important modules in Cockpit are: Collections and Galleries.

Continue reading %Introducing CockpitCMS – a CMS for Developers%

Link
Cal EvansInterview with Jacob Mather (28.1.2015, 15:41 UTC)

Twitter: @thejmather

Show Notes

Link
thePHP.ccPHP breaks backwards compatibility (28.1.2015, 07:00 UTC)
Link
Evert PotA new URI handling library for PHP. (27.1.2015, 22:43 UTC)

The last little while have been all about URI's for me. I've been engaged in discussions about the upcoming 'PSR-7', I started working on a sort of 'hypermedia'-focussed router/repository type of system, and I recently ran into a bug in sabredav that forced me to get a better understanding of the URI specification, in particular on the topics of 'normalization' and resolving relative URLs.

All of this accumulated in a set of functions of operations I commonly need, which I'm now releasing as an open source library.

Yup! Yet another URI library. All this library is though, is a collection of 5 php functions:

  • resolve to resolve relative uris.
  • normalize to aid in comparing uris.
  • parse, which mostly works like PHP's parse_url.
  • build to do the exact opposite of parse.
  • split to easily get the 'dirname' and 'basename' of a URL without all the problems those two functions have.

The bug I ran into with sabredav, had to do with a client using a different encoding of a URI than I expected to. After diving into rfc3986, I realized that an application comparing uris cannot simply compare using strcmp or ===.

For example, the following uris are all identical:

  • http://example.org/~foo/
  • HTTP://example.ORG/~foo/
  • http://example.org:80/~foo/
  • http://example.org/%7Efoo/
  • http://example.org/%7efoo/
  • http://example.org/bar/./../~foo/

The new normalize function can take any of these previous URIs, and will turn it into the first.

<?php

use Sabre\Uri;

$input = 'HTTP://example.ORG:80/bar/./../%7efoo/';

echo Uri::normalize($input);
// output: http://example.org/~foo/

?>

I hope it's useful for others.

Links:

Link
Derick RethansQuestions from the Field: Should I Escape My Input, And If So, How? (27.1.2015, 09:03 UTC)

Questions from the Field: Should I Escape My Input, And If So, How?

At last weekend's PHP Benelux I gave a tutorial titled "From SQL to NoSQL". Large parts of the tutorial covered using MongoDB—how to use it from PHP, schema design, etc. I ran a little short of time, and since then I've been getting some questions. One of them being: "Should I escape my input, and if so, how?". Instead of trying to cram my answer in 140 characters on Twitter, I thought it'd be wise to reply with this blog post.

The short answer is: yes, you do need to escape.

The longer answer is a bit more complicated.

Unlike with SQL, inserting, updating and deleting data, as well as querying data, does not require the creation of strings in MongoDB. All data is always used as a variable or a constant. Take for example:

<?php
$c = (new MongoClient())->demo->col;
$c->insert( [ 'name' => $_GET['name'] ] );
?>

Because we don't need to create a string with the full insert statement, there is no need to escape with ' to prevent issues like SQL injections. The context in which variables are used is immediately clear.

But be aware that PHP's request parameters (GET, POST, COOKIE, and others) allow you to send not only scalar values, but also arrays. If we take the example code from above in mind, and request the URL http://localhost/script.php?name[first]=Derick&name[last]=Rethans, we end up inserting the following document into the collection:

[ 'name' => [
        'first' => 'Derick',
        'last' => 'Rethans'
] ]

And this is probably not what you had in mind.

The same trick is possible when doing queries. Look at this code:

<?php
$c = (new MongoClient())->demo->col;

$r = $c->findOne( [
        'user_id' => $_GET['uid'],
        'password' => $_GET['password']
] );
?>

If we now would request the URL http://localhost/script.php?uid=3&password[$neq]=foo we end up doing the following query:

<?php
$c = (new MongoClient())->demo->col;

$r = $c->findOne( [
        'user_id' => '3',
        'password' => [ '$neq' => 'foo' ]
] );
?>

The password clause in that query, will likely always match. Of course, if you are not storing passwords as a hash, you have other problems too! This is just a simple example to illustrate the problem.

This same example highlights the second issue - that is that all request parameters are always represented by strings in PHP. Hence my use of '3' instead of 3 in the above example. MongoDB treats '3' and 3 differently while matching, and searching for 'user_id' => '3' will not find documents where 3 is stored as a number. I wrote more extensively about that before.

So although MongoDB's query language does not require you to build strings, and hence "escape" input, it is required that you either make sure that the data is of the correct data type. For example you can do:

<?php
$c = (new MongoClient())->demo->col;

$r = $c->findOne( [
        'user_id' => (int) $_GET['uid'],
        'password' => (string) $_GET['password']
] );
?>

For scalar values, often a cast like I've done above, is the easiest, but you might end up converting an array to the string 'Array' or the number 1.

In most cases, it means that if you want to do things right, you will need to check the data types of GET/POST/COOKIE parameters, and cast, convert, or bail out as appropriate.

Link
LinksRSS 0.92   RDF 1.
Atom Feed   100% Popoon
PHP5 powered   PEAR
ButtonsPlanet PHP   Planet PHP
Planet PHP