One conference per day, for one year (2017)

My self-assigned challenge for 2017 was to watch at least one conference per day, for one year. That’s the first time I try this challenge. Let’s dive in for a recap.

267 conferences

In some way, I failed the challenge because I’ve been able to watch only 267 conferences. With an average of 34 minutes per conference, I’ve watched 9078 minutes, or 151 hours of freely available conferences online. Why did I fail to watch 365 of them? Because my first kid was 1.5 years in January 2017, a new little lady came in December 2017, I got a new job, I travelled for my job, I gave talks, I maintain important open source projects requiring lot of time, I’m building my own self-sufficient ecological house, the vegetable garden requires many hours, I watch other videos, and because I’m lazy sometimes. Most of the time, I was able to watch 2 or 3 conferences in a row.

Where to find the resources?

All these conferences are freely available online, on YouTube, or on Vimeo, for most of them. The channel I mostly watch are the following:

It’s very Computer Science centric as you might have noticed, and it targets Rust, C++, Elm, LLVM, or Web technologies (JS, CSS…), but not only, you can find Haskell or Clojure sometimes.

My best-of list

In March 2017, more and more people were questionning me, and asked for sharing. I then decided to start a playlist of my “best-of” conferences. I’ve added 78 conferences in 2017, and 3 new conferences have been added since then.

Thumnails of my “best-of” 2017

Thoughts and conclusion

The challenge was sometimes easy and relaxing, or it was very hard to understand everything especially at 2am after a long day (looking at you CppCon). But it has been a very enjoyable way to learn a lot in a very short period of time. Many speakers are talented, and listening to them is a real pleasure. Some others are just… let’s say unprepared, and it’s good to stop and jump onto another talk. It’s also a good way to get inspired by technologies you don’t necessarily know (for instance, I’m not a big fan of Clojure, but some projects are really inspiring, like Proto REPL).

Sometimes I tweeted about the talk I watched, and it was quite appreciated too. I reckon because it’s a fun and an easy way to learn, especially with the help of video platforms like Youtube.

Am I going to continue this challenge in 2018? Yes! But maybe not at this frequency. It’s now part of my routine to watch conferences many times per week. I like it. I don’t want to stop.

As a closing note, I would like to thank every speakers, and more importantly, every conference organizer. You are doing an amazing job: From the program, to the event, to the final sharing on Internet with everyone. Most of you are volunteers. I know the work it represents. You are producing extremely valuable resources. Thank you!

Random thoughts about `::class` in PHP

The special ::class constant allows for fully qualified class name resolution at compile, this is useful for namespaced classes.

I’m quoting the PHP manual. But things can be funny sometimes. Let’s go through some examples.

  • use A\B as C;
    
    $_ = C::class;

    resolves to A\B, which is perfect 🙂

  • class C
    {
        public function f()
        {
            $_ = self::class;
        }
    }

    resolves to C, which is perfect 😀

  • class C { }
    
    class D extends C
    {
        public function f()
        {
            $_ = parent::class;
        }
    }

    resolves to C, which is perfect 😄

  • class C
    {
        public static function f()
        {
            $_ = static::class;
        }
    }
    
    class D extends C { }
    
    D::f();

    resolves to D, which is perfect 😍

  • 'foo'::class

    resolves to 'foo', which is… huh? 🤨

  • "foo"::class

    resolves to 'foo', which is… expected somehow 😕

  • $a = 'oo';
    "f{$a}"::class

    generates a parse error 🙃

  • PHP_VERSION::class

    resolves to 'PHP_VERSION', which is… strange: It resolves to the fully qualified name of the constant, not the class 🤐

::class is very useful to get rid off of the get_class or the get_called_class functions, or even the get_class($this) trick. This is something truly useful in PHP where entities are referenced as strings, not as symbols. ::class on constants makes sense, but the name is no longer relevant. And finally, ::class on single quote strings is absolutely useless; on double quotes strings it is a source of error because the value can be dynamic (and remember, ::class is resolved at compile time, not at run time).

atoum supports TeamCity

atoum is a popular PHP test framework. TeamCity is a Continuous Integration and Continuous Delivery software developed by Jetbrains. Despites atoum supports many industry standards to report test execution verdicts, TeamCity uses its own non-standard report, and thus atoum is not compatible with TeamCity… until now.

icon_TeamCity

The atoum/teamcity-extension provides TeamCity support inside atoum. When executing tests, the reported verdicts are understandable by TeamCity, and activate all its UI features.

Install

If you have Composer, just run:

$ composer require atoum/teamcity-extension '~1.0'

From this point, you need to enable the extension in your .atoum.php configuration file. The following example forces to enable the extension for every test execution:

$extension = new atoum\teamcity\extension($script);
$extension->addToRunner($runner);

The following example enables the extension only within a TeamCity environment:

$extension = new atoum\teamcity\extension($script);
$extension->addToRunnerWithinTeamCityEnvironment($runner);

This latter installation is recommended. That’s it 🙂.

Glance

The default CLI report looks like this:

Default atoum CLI report

The TeamCity report looks like this in your terminal (note the TEAMCITY_VERSION variable as a way to emulate a TeamCity environment):

TeamCity report inside the terminal

Which is less easy to read. However, when it comes into TeamCity UI, we will have the following result:

TeamCity running atoum

We are using it at Automattic. Hope it is useful for someone else!

If you find any bugs, or would like any other features, please use Github at the following repository: https://github.com/Hywan/atoum-teamcity-extension/.

Export functions in PHP à la Javascript

Warning: This post is totally useless. It is the result of a fun private company thread.

Export functions in Javascript

In Javascript, a file can export functions like this:

export function times2(x) {
    return x * 2;
}

And then we can import this function in another file like this:

import {times2} from 'foo';

console.log(times2(21)); // 42

Is it possible with PHP?

Export functions in PHP

Every entity is public in PHP: Constant, function, class, interface, or trait. They can live in a namespace. So exporting functions in PHP is absolutely useless, but just for the fun, let’s keep going.

A PHP file can return an integer, a real, an array, an anonymous function, anything. Let’s try this:

<?php

return function (int $x): int {
    return $x * 2;
};

And then in another file:

<?php

$times2 = require 'foo.php';
var_dump($times2(21)); // int(42)

Great, it works.

What if our file returns more than one function? Let’s use an array (which has most hashmap properties):

<?php

return [
    'times2' => function (int $x): int {
        return $x * 2;
    },
    'answer' => function (): int {
        return 42;
    }
];

To choose what to import, let’s use the list intrinsic. It has several forms: With or without key matching, long (list(…)) and short syntax ([…]). Because we are modern, we will use the short syntax with key matching to selectively import functions:

<?php

['times2' => $mul] = require 'foo.php';

var_dump($mul(21)); // int(42)

Notice that times2 has been aliased to $mul. What a feature!

Is it useful? Absolutely not. Is it fun? For me it is.

Finite-State Machine as a Type System illustrated with a store product

Hello fellow coders!

In this article, I would like to talk about how to implement a Finite-State Machine (FSM) with the PHP type system. The example is a store product (in an e-commerce solution for instance), something we are likely to meet once in our lifetime. Our goal is to simply avoid impossible states and transitions.

I am in deep love with Type theory, however I will try to keep the formulas away from this article to focus on the code. Moreover, you might be aware that the PHP runtime type system is somewhat very permissive and “poor” (this is not a formal definition), hopefully some tricks can help us to express nice constraints.

The Product FSM

A product in a store might have the following states:

  • Active: Can be purchased,
  • Inactive: Has been cancelled or discontinued (a discontinued product can no longer be purchased),
  • Purchased and renewable,
  • Purchased and not renewable,
  • Purchased and cancellable.

The transitions between these states can be viewed as a Finite-State Machine (FSM).

AyxEp2j8B4hCLIZEI4p9By_CIrT8IymfJkNYYjQALT3LjLDmv784qquALWfA1QL5oHc9nQbAN4u8mQBKlDHo9QWoPv18Vbvogcv-Mfe2GZrKWmj8EZaHA9-ZnEMC8GG0
Product FSM (editable source).

We read this graph as: A product is in the state A. If the purchase action is called, then it transitions to the state B. If the once-off purchase action is called, then it transitions to the state C. From the state B, if the renew action is called, it remains in the same state. If the cancel action is called, it transitions to the D state. Same for the C to D states.

Our goal is to respect this FSM. Invalid actions must be impossible to do.

Finite-State Machine as a Type System

Having a FSM is a good thing to define the states and the transitions between them: It is formal and clear. However, it is tested at runtime, not at compile-time, i.e. if statements are required to test if the state of a product can transition into another state, or else throw an exception, and this is decided at runtime. Note that PHP does not really have a compile-time because it is an online compiler (learn more by reading Tagua VM, a safe PHP virtual machine, at slide 29). Our goal is to prevent illegal/invalid states at parse-/compile-time so that the PHP virtual machine, IDE or static analysis tools can prove the state of a product without executing PHP code.

Why is this important? Imagine that we decide to change a product to be once-off purchasable instead of purchasable, then we can no longer renew it. We replace an interface on this product, and boom, the IDE tells us that the code is broken in x places. It detects impossible scenarios ahead of code execution.

No more talking. Here is the code.

The mighty product

/**
 * A product.
 */
interface Product { }

A product is a class implementing the Product interface. It allows to type a generic product, with no regards about its state.

Active and inactive

/**
 * A product that is active.
 */
interface Active extends Product
{
    public function getProduct(): self;
}

/**
 * A product that has been cancelled, or not in stock.
 */
interface Inactive extends Product
{
    public function getProduct(): self;
}

The Active and Inactive interfaces are useful to create constraints such as:

  • A product can be purchased only if it is active, and
  • A product is inactive if and only if it has been cancelled,
  • To finally conclude that an inactive product can no longer be purchased, nor renewed, nor cancelled.

Basically, it defines the axiom (initial state) and the final states of our FSM.

The getProduct(): self trick will make sense later. It helps to express the following constraint: “A valid product cannot be invalid, and vice-versa”, i.e. both interfaces cannot be implemented by the same value.

Purchase, renew, and cancel

/**
 * A product that can be purchased.
 */
interface Purchasable extends Active
{
    public function purchase(): Renewable;
}

Only an active product can be purchased. The action is purchase and it generates a product that is renewable. purchase transitions from the state A to B (regarding the graph above).

/**
 * A product that can be cancelled.
 */
interface Cancellable extends Active
{
    public function cancel(): Inactive;
}

Only an active product can be cancelled. The action is cancel and it generates an inactive product, so it transitions from the state B to D.

/**
 * A product that can be renewed.
 */
interface Renewable extends Cancellable
{
    public function renew(): self;
}

A renewable product is also cancellable. The action is renew and this is a reflexive transition from the state B to B.

/**
 * A product that can be once-off purchased, i.e. it can be purchased but not
 * renewed.
 */
interface PurchasableOnce extends Active
{
    public function purchase(): Cancellable;
}

Finally, a once-off purchasable product has one action: purchase that produces a Cancellable product, and it transitions from the state A to C.

Take a breath

AyxEp2j8B4hCLIZEI4p9By_CIrT8IymfJkNYAYv9B4bLS4mkoInBLQZcKW22QArO1LrTEmL7CCyHp7PIi59G2YWjIiv8B4vCoacriYg0S5ALmAgS4Ag2KlDIoo5gYa1C9IHZdD6CySzBHZ6g5kOWpxn4P2T1Z7S1wNPE1Eh9oO5Oa0pcG6nm9g2c5W
Detailed product FSM (editable source).

So far we have defined interfaces, but the FSM is not implemented yet. Interfaces only define constraints in our type system. An interface provides a constraint but also defines type capabilities: What operations can be performed on a value implementing a particular interface.

SecretProduct

Let’s consider the SecretProduct as a new super secret product that will revolutionise our store:

/**
 * The `SecretProduct` class is:
 *
 *   * A product,
 *   * Active,
 *   * Purchasable.
 *
 * Note that in this implementation, the `SecretProduct` instance is mutable: Every
 * action happens on the same `SecretProduct` instance. It makes sense because
 * having 2 instances of the same product with different states might be error-prone
 * in most scenarios.
 */
class SecretProduct implements Active, Purchasable
{
    public function getProduct(): Active
    {
        return $this;
    }

    /**
     * Purchase the product will return an active product that is renewable,
     * and also cancellable.
     */
    public function purchase(): Renewable
    {
        return new class ($this->getProduct()) implements Renewable {
            protected $product;

            public function __construct(SecretProduct $product)
            {
                $this->product = $product;
                // Do the purchase.
            }

            public function getProduct(): Active
            {
                return $this->product;
            }

            public function renew(): Renewable
            {
                // Do the renew.
                return $this;
            }

            public function cancel(): Inactive
            {
                return new class ($this->getProduct()) implements Inactive {
                    protected $product;

                    public function __construct(SecretProduct $product)
                    {
                        $this->product = $product;
                        // Do the cancel.
                    }

                    public function getProduct(): Inactive
                    {
                        return $this->product;
                    }
                };
            }
        };
    }
}

The SecretProduct is a product that is active and purchasable. PHP verifies that the Active::getProduct method is implemented, and that the Purchasable::purchase method is implemented too.

When this latter is called, it returns an object implementing the Renewable interface (which is also a cancellable active product). The object in this context is an instance of an anonymous class implementing the Renewable interface. So the Active::getProduct, Renewable::renew, and Cancellable::cancel methods must be implemented.

Having an anonymous class is not required at all, this is just simpler for the example. A named class may even be better from the testing point of view.

Note that:

  • The real purchase action is performed in the constructor of the anonymous class: This is not a hard rule, this is just convenient; it can be done in the method before returning the new instance,
  • The real renew action is performed in the renew method before returning $this,
  • And the real cancel action is performed in… we have to dig a little bit more (the principle is exactly the same though):
    • The Cancellable::cancel method must return an object implementing the Inactive interface.
    • It generates an instance of an anonymous class implementing the Inactive interface, and the real cancel action is done in the constructor.

Assert possible and impossible actions

Let’s try some valid and invalid actions. Those followings are possible actions:

assert((new SecretProduct())->purchase()                             instanceof Product);
assert((new SecretProduct())->purchase()->renew()                    instanceof Product);
assert((new SecretProduct())->purchase()->cancel()                   instanceof Product);
assert((new SecretProduct())->purchase()->renew()->renew()->cancel() instanceof Product);

It is possible to purchase a product, then renew it zero or many times, and finally to cancel it. It matches the FSM!

Those followings are impossible actions:

(new SecretProduct())->renew();
(new SecretProduct())->cancel();
(new SecretProduct())->purchase()->cancel()->purchase();
(new SecretProduct())->purchase()->cancel()->renew();
(new SecretProduct())->purchase()->purchase();
(new SecretProduct())->purchase()->cancel()->cancel();

It is impossible:

  • To renew or to cancel a product that has not been purchased,
  • To purchase or renew a product that has been cancelled,
  • To purchase a product more than once,
  • To cancel a product more than once.

Those followings are impossible implementations:

class SecretProduct implements Active, Purchasable, PurchasableOnce { }

A product cannot be purchasable and once-off purchasable at the same time, because Purchasable::purchase is not compatible with PurchasableOnce::purchase.

class SecretProduct implements Inactive, Cancellable { }

An inactive product cannot be purchased nor renewed nor cancelled because Active::getProduct and Inactive::getProduct are not compatible.

Wow, that’s great garantees isn’t it? PHP will raise fatal errors for impossible actions or impossible states. No warnings or notices: Fatal errors. Most of them are correctly inferred by IDE, so… follow the red crosses in your IDE.

Restoring a product

One major thing is missing: The state of a product is stored in the database. When loading the product, we must be able to get an instance of a product at its previous state. To avoid repeating code, we will use traits. Rebuilding the state of a product is “just” (it really is) a composition of traits.

Note: In these examples, we are using anonymous classes and traits. It is possible to achieve the same behavior with final named classes. Also we are using a repository, which is convenient for this article, but not necessarily the best solution.

Repository

The following ProductRepository\load function is just here to give you an idea of how it works.

namespace ProductRepository;

function load(int $id, string $state): Product
{
    // Load the product from the database with `$id`.
    //
    // The states can be `Renewable`, `Cancellable`, or `Inactive` (check
    // the FSM to double-check). Products that have not been purchased
    // are not in the database.

    // Fake minimal active product.
    $product = new class implements Active {
        public function getProduct(): Active {
            return $this;
        }
    };

    switch ($state) {
        // State B.
        case Renewable::class:
            return new class ($product) implements Renewable {
                use ActiveProduct;
                use RenewableProduct;
                use CancellableProduct;
            };

        // State C.
        case Cancellable::class:
            return new class ($product) implements Cancellable {
                use ActiveProduct;
                use CancellableProduct;
            };

        // State D.
        case Inactive::class:
            return new class ($product) implements Inactive {
                use InactiveProduct;
            };

        // Invalid state.
        default:
            throw new RuntimeException('Invalid product state.');
    }
}

Traits

The code must look familiar because this is just a split from the SecretProduct implementation.

trait ActiveProduct
{
    protected $product;

    public function __construct(Product $product)
    {
        $this->product = $product;
    }

    public function getProduct(): Active
    {
        return $this->product;
    }
}

trait RenewableProduct
{
    public function renew(): Renewable
    {
        // Do the renew.
        return $this;
    }
}

trait CancellableProduct
{
    public function cancel(): Inactive
    {
        return new class ($this->getProduct()) implements Inactive {
            protected $product;

            public function __construct(Product $product)
            {
                $this->product = $product;
                // Do the cancel.
            }

            public function getProduct(): Inactive
            {
                return $this->product;
            }
        };
    }
}

trait InactiveProduct
{
    protected $product;

    public function __construct(Product $product)
    {
        $this->product = $product;
    }

    public function getProduct(): Inactive
    {
        return $this->product;
    }
}

Assert possible and impossible actions

The possible actions are:

$product = ProductRepository\load(42, Renewable::class);

assert($product           instanceof Product);
assert($product->renew()  instanceof Product);
assert($product->cancel() instanceof Product);

Product 42 is assumed to be in the state B (Renewable::class), so we can renew and cancel it.

Those followings are impossible actions:

$product = ProductRepository\load(42, Renewable::class);

$product->purchase();
$product->cancel()->cancel();

It is impossible to purchase the product 42 because it is in state B, so it has already been purchased. It is impossible to cancel a product twice.

Same garantees apply here!

Conclusion

It is possible to re-implement SecretProduct with the traits we have defined for the ProductRepository, or to use named classes. I let this as an easy wrap up exercise for the reader.

The real conclusion is that we have successfully implemented the Finite-State Machine of a product with a Type System. It is impossible to have an invalid implementation that violates the constraints, such as an inactive renewable product. PHP detects it immediately at runtime. Invalid actions are also impossible, such as purchasing a product twice, or renewing a once-off purchased product. It is also detected by PHP.

All violations take the form of PHP fatal errors.

The product repository is an example of how to restore a product at a particular state, with the help of the defined interfaces, and new small and simple traits.

One more thing

It is possible to integrate product categories in this type system (like bundles). It is more complex, but possible.

I would highly recommend these following readings:

I would like to particularly emphasize a paragraph from the first article:

So what is a type? The only true definition is this: a type is a label used by a type system to prove some property of the program’s behavior. If the type checker can assign types to the whole program, then it succeeds in its proof; otherwise it fails and points out why it failed.

Seeing types as labels is a very smart way of approaching them.

I would like to thanks Marco Pivetta for the reviews!

sabre/katana

sabre/katana's logo
Project’s logo.

What is it?

sabre/katana is a contact, calendar, task list and file server. What does it mean? Assuming nowadays you have multiple devices (PC, phones, tablets, TVs…). If you would like to get your address books, calendars, task lists and files synced between all these devices from everywhere, you need a server. All your devices are then considered as clients.

But there is an issue with the server. Most of the time, you might choose Google or maybe Apple, but one may wonder: Can we trust these servers? Can we give them our private data, like all our contacts, our calendars, all our photos…? What if you are a company or an association and you have sensitive data that are really private or strategic? So, can you still trust them? Where the data are stored? Who can look at these data? More and more, there is a huge need for “personal” server.

Moreover, servers like Google or Apple are often closed: You reach your data with specific clients and they are not available in all platforms. This is for strategic reasons of course. But with sabre/katana, you are not limited. See the above schema: Firefox OS can talk to iOS or Android at the same time.

sabre/katana is this kind of server. You can install it on your machine and manage users in a minute. Each user will have a collection of address books, calendars, task lists and files. This server can talk to a loong list of devices, mainly thanks to a scrupulous respect of industrial standards:

  • Mac OS X:
    • OS X 10.10 (Yosemite),
    • OS X 10.9 (Mavericks),
    • OS X 10.8 (Mountain Lion),
    • OS X 10.7 (Lion),
    • OS X 10.6 (Snow Leopard),
    • OS X 10.5 (Leopard),
    • BusyCal,
    • BusyContacts,
    • Fantastical,
    • Rainlendar,
    • ReminderFox,
    • SoHo Organizer,
    • Spotlife,
    • Thunderbird ,
  • Windows:
    • eM Client,
    • Microsoft Outlook 2013,
    • Microsoft Outlook 2010,
    • Microsoft Outlook 2007,
    • Microsoft Outlook with Bynari WebDAV Collaborator,
    • Microsoft Outlook with iCal4OL,
    • Rainlendar,
    • ReminderFox,
    • Thunderbird,
  • Linux:
    • Evolution,
    • Rainlendar,
    • ReminderFox,
    • Thunderbird,
  • Mobile:
    • Android,
    • BlackBerry 10,
    • BlackBerry PlayBook,
    • Firefox OS,
    • iOS 8,
    • iOS 7,
    • iOS 6,
    • iOS 5,
    • iOS 4,
    • iOS 3,
    • Nokia N9,
    • Sailfish.

Did you find your device in this list? Probably yes 😉.

sabre/katana sits in the middle of all your devices and synced all your data. Of course, it is free and open source. Go check the source!

List of features

Here is a non-exhaustive list of features supported by sabre/katana. Depending whether you are a user or a developer, the features that might interest you are radically not the same. I decided to show you a list from the user point of view. If you would like to get a list from the developer point of view, please see this exhaustive list of supported RFC for more details.

Contacts

All usual fields are supported, like phone numbers, email addresses, URLs, birthday, ringtone, texttone, related names, postal addresses, notes, HD photos etc. Of course, groups of cards are also supported.

My card on Mac OS X
My card inside the native Contact application of Mac OS X.
My card on Firefox OS
My card inside the native Contact application of Firefox OS.

My photo is not in HD, I really have to update it!

Cards can be encoded into several formats. The most usual format is VCF. sabre/katana allows you to download the whole address book of a user as a single VCF file. You can also create, update and delete address books.

Calendars

A calendar is just a set of events. Each event has several properties, such as a title, a location, a date start, a date end, some notes, URLs, alarms etc. sabre/katana also support recurring events (“each last Monday of the month, at 11am…”), in addition to scheduling (see bellow).

My calendars on Mac OS X
My calendars inside the native Calendar application of Mac OS X.
My calendars on Firefox OS
My calendars inside the native Calendar application of Firefox OS.

Few words about calendar scheduling. Let’s say you are organizing an event, like New release (we always enjoy release day!). You would like to invite several people but you don’t know if they could be present or not. In your event, all you have to do is to add attendees. How are they going to be notified about this event? Two situations:

  1. Either attendees are registered on your sabre/katana server and they will receive an invite inside their calendar application (we call this iTIP),
  2. Or they are not registered on your server and they will receive an email with the event as an attached file (we call this iMIP). All they have to do is to open this event in their calendar application.
Typical mail to invite an attendee to an event
Invite an attendee by email because she is not registered on your sabre/katana server.

Notice the gorgeous map embedded inside the email!

Once they received the event, they can accept, decline or “don’t know” (they will try to be present at) the event.

Receive an invite to an event
Receive an invite to an event. Here: Gordon is inviting Hywan. Three choices for Hywan:

, or

.
Status of all attendees
Hywan has accepted the event. Here is what the event looks like. Hywan can see the response of each attendees.
Notification from attendees
Gordon is even notified that Hywan has accepted the event.

Of course, attendees will be notified too if the event has been moved, canceled, refreshed etc.

Calendars can be encoded into several formats. The most usal format is ICS. sabre/katana allows you to download the whole calendar of a user as a single ICS file. You can also create, update and delete calendars.

Task lists

A task list is exactly like a calendar (from a programmatically point of view). Instead of containg event objects, it contains todo objects.

sabre/katana supports group of tasks, reminder, progression etc.

My task lists on Mac OS X
My task lists inside the native Reminder application of Mac OS X.

Just like calendars, task lists can be encoded into several formats, whose ICS. sabre/katana allows you to download the whole task list of a user as a single ICS file. You can also create, update and delete task lists.

Files

Finally, sabre/katana creates a home collection per user: A personal directory that can contain files and directories and… synced between all your devices (as usual 😄).

sabre/katana also creates a special directory called public/ which is a public directory. Every files and directories stored inside this directory are accessible to anyone that has the correct link. No listing is prompted to protect your public data.

Just like contact, calendar and task list applications, you need a client application to connect to your home collection on sabre/katana.

Connect to a server in Mac OS X
Connect to a server with the Finder application of Mac OS X.

Then, your public directory on sabre/katana will be a regular directory as every other.

List of my files
List of my files, right here in the Finder application of Mac OS X.

sabre/katana is able to store any kind of files. Yes, any kinds. It’s just files. However, it white-lists the kind of files that can be showed in the browser. Only images, audios, videos, texts, PDF and some vendor formats (like Microsoft Office) are considered as safe (for the server). This way, associations can share musics, videos or images, companies can share PDF or Microsoft Word documents etc. Maybe in the future sabre/katana might white-list more formats. If a format is not white-listed, the file will be forced to download.

How is sabre/katana built?

sabre/katana is based on two big and solid projects:

  1. sabre/dav,
  2. Hoa.

sabre/dav is one of the most powerful CardDAV, CalDAV and WebDAV framework in the planet. Trusted by the likes of Atmail, Box, fruux and ownCloud, it powers millions of users world-wide! It is written in PHP and is open source.

Hoa is a modular, extensible and structured set of PHP libraries. Fun fact: Also open source, this project is also trusted by ownCloud, in addition to Mozilla, joliCode etc. Recently, this project has recorded more than 600,000 downloads and the community is about to reach 1000 people.

sabre/katana is then a program based on sabre/dav for the DAV part and Hoa for everything else, like the logic code inside the sabre/dav‘s plugins. The result is a ready-to-use server with a nice interface for the administration.

To ensure code quality, we use atoum, a popular and modern test framework for PHP. So far, sabre/dav has more than 1000 assertions.

Conclusion

sabre/katana is a server for contacts, calendars, task lists and files. Everything is synced, everytime and everywhere. It perfectly connects to a lot of devices on the market. Several features we need and use daily have been presented. This is the easiest and a secure way to host your own private data.

Go download it!

RFCs should provide executable test suites

Recently, I implemented xCal and xCard formats inside the sabre/dav libraries. While testing the different RFCs against my implementation, several errata have been found. This article, first, quickly list them and, second, ask questions about how such errors can be present and how they can be easily revealed. If reading my dry humor about RFC errata is boring, the Sections 3, 4 and 5 are more interesting. The whole idea is: Why RFCs do not provide executable test suites?

What is xCal and xCard?

The Web is a read-only media. It is based on the HTTP protocol. However, there is the WebDAV protocol, standing for Web Distributed Authoring and Versioning. This is an extension to HTTP. Et voilà ! The Web is a read and write media. WebDAV is standardized in RFC2518 and RFC4918.

Based on WebDAV, we have CalDAV and CardDAV, respectively for reading and writing calendars and addressbooks. They are standardized in RFC4791, RFC6638 and RFC6352. Good! But these protocols only explain how to read and write, not how to represent a real calendar or an addressbook. So let’s leave protocols for formats.

The iCalendar format represents calendar events, like events (VEVENT), tasks (VTODO), journal entry (VJOURNAL, very rare…), free/busy time (VFREEBUSY) etc. The vCard format represents cards. The formats are very similar and share a common ancestry: This is a horrible line-, colon- and semicolon-, randomly-escaped based format. For instance:

BEGIN:VCALENDAR
VERSION:2.0
CALSCALE:GREGORIAN
PRODID:-//Example Inc.//Example Calendar//EN
BEGIN:VEVENT
DTSTAMP:20080205T191224Z
DTSTART;VALUE=DATE:20081006
SUMMARY:Planning meeting
UID:4088E990AD89CB3DBB484909
END:VEVENT
END:VCALENDAR

Horrible, yes. You were warned. These formats are standardized in several RFCs, to list some of them: RFC5545, RFC2426 and RFC6350.

This format is impossible to read, even for a computer. That’s why we have jCal and jCard, which are respectively another representation of iCalendar and vCard but in JSON. JSON is quite popular in the Web today, especially because it eases the manipulation and exchange of data in Javascript. This is just a very simple, and —from my point of view— human readable, serialization format. jCal and jCard are respectively standardized in RFC7265 and RFC7095. Thus, the equivalent of the previous iCalendar example in jCal is:

[
    "vcalendar",
    [
        ["version", {}, "text", "2.0"],
        ["calscale", {}, "text", "GREGORIAN"],
        ["prodid", {}, "text", "-\/\/Example Inc.\/\/Example Calendar\/\/EN"]
    ],
    [
        [
            "vevent",
            [
                ["dtstamp", {}, "date-time", "2008-02-05T19:12:24Z"],
                ["dtstart", {}, "date", "2008-10-06"],
                ["summary", {}, "text", "Planning meeting"],
                ["uid", {}, "text", "4088E990AD89CB3DBB484909"]
            ]
        ]
    ]
]

Much better. But this is JSON, which is a rather loose format, so we also have xCal and xCard another representation of iCalendar and vCard but in XML. They are standardized in RFC6321 and RFC6351. The same example in xCal looks like this:

<icalendar xmlns="urn:ietf:params:xml:ns:icalendar-2.0">
 <vcalendar>
  <properties>
   <version>
    <text>2.0text>
   version>
   <calscale>
    <text>GREGORIANtext>
   calscale>
   <prodid>
    <text>-//Example Inc.//Example Calendar//ENtext>
   prodid>
  properties>
  <components>
   <vevent>
    <properties>
     <dtstamp>
      <date-time>2008-02-05T19:12:24Zdate-time>
     dtstamp>
     <dtstart>
      <date>2008-10-06date>
     dtstart>
     <summary>
      <text>Planning meetingtext>
     summary>
     <uid>
      <text>4088E990AD89CB3DBB484909text>
     uid>
    properties>
   vevent>
  components>
 vcalendar>
icalendar>

More semantics, more meaning, easier to read (from my point of view), namespaces… It is very easy to embed xCal and xCard inside other XML formats.

Managing all these formats is an extremely laborious task. I suggest you to take a look at sabre/vobject (see the Github repository of sabre/vobject). This is a PHP library to manage all the weird formats. The following example shows how to read from iCalendar and write to jCal and xCal:

// Read iCalendar.
$document = Sabre\VObject\Reader::read($icalendar);

// Write jCal.
echo Sabre\VObject\Writer::writeJson($document);

// Write xCal.
echo Sabre\VObject\Writer::writeXml($document);

Magic when you know the complexity of these formats (in both term of parsing and validation)!

List of errata

Now, let’s talk about all the errata I submited recently:

The 2 last ones are reported, not yet verified.

4241, 4243 and 4246 are just typos in examples. “just” is a bit of an under-statement when you are reading RFCs for days straight, you have 10 of them opened in your browser and trying to figure out how everything fits together and if you are doing everything correctly. Finding typos at that point in your process can be very confusing…

4247 is more subtle. The RFC about xCard comes with an XML Schema. That’s great! It will help us to test our documents and see if they are valid or not! No? No.

Most of the time, I try to relax and deal with the incoming problems. But the date and time format in iCalendar, vCard, jCal, jCard, xCal and xCard can make my blood boil in a second. In what world, exactly, --10 or ---28 is a conceivable date and time format? How long did I sleep? “Well” — was I saying to myself, “do not make a drama, we have the XML Schema!”. No. Because there is an error in the schema. More precisely, in a regular expression:

value-time = element time {
    xsd:string { pattern = "(\d\d(\d\d(\d\d)?)?|-\d\d(\d\d?)|--\d\d)"
                         ~ "(Z|[+\-]\d\d(\d\d)?)?" }
}

Did you find the error? (\d\d?) is invalid, this is (\d\d)?. Don’t get me wrong: Everyone makes mistakes, but not this kind of error. I will explain why in the next section.

4245 is not an editorial error but a technical one, under review.

4261 is crazy. It deserves a whole sub-section.

Welcome in the crazy world of date and time formats

There are two major popular date and time format: RFC2822 and ISO.8601. Examples:

  • Fri, 27 Feb 2015 16:06:58 +0100 and
  • 2015-02-27T16:07:16+01:00.

The second one is a good candidate for a computer representation: no locale, only digits, all information are present…

Maybe you noticed there is no link on ISO.8601. Why? Because ISO standards are not free and I don’t want to pay 140€ to buy a standard…

The date and time format adopted by iCalendar and vCard (and the rest of the family) is ISO.8601.2004. I cannot read it. However, since we said in xCard we have an XML Schema; we can read this (after having applied erratum 4247):

# 4.3.1
value-date = element date {
    xsd:string { pattern = "\d{8}|\d{4}-\d\d|--\d\d(\d\d)?|---\d\d" }
  }

# 4.3.2
value-time = element time {
    xsd:string { pattern = "(\d\d(\d\d(\d\d)?)?|-\d\d(\d\d)?|--\d\d)"
                         ~ "(Z|[+\-]\d\d(\d\d)?)?" }
  }

# 4.3.3
value-date-time = element date-time {
    xsd:string { pattern = "(\d{8}|--\d{4}|---\d\d)T\d\d(\d\d(\d\d)?)?"
                         ~ "(Z|[+\-]\d\d(\d\d)?)?" }
  }

# 4.3.4
value-date-and-or-time = value-date | value-date-time | value-time

Question: --10 is October or 10 seconds?

--10 can fit into value-date and value-time:

  • From value-date, the 3rd element in the disjunction is --\d\d(\d\d)?, so it matches --10,
  • From value-time, the last element in the first disjunction is --\d\d, so it matches --10.

If we have a date-and-or-time value, value-date comes first, so --10 is always October. Nevertheless, if we have a time value, --10 is 10 seconds. Crazy now?

Oh, and XML has its own date and time format, which is well-defined and standardized. Why should we drag this crazy format along?

Oh, and I assume every format depending on ISO.8601.2004 has this bug. But I am not sure because ISO standards are not free.

How can RFCs have such errors?

So far, RFCs are textual standards. Great. But they are just text. Written by humans, and thus they are subject to errors or failures. It is even error-prone. I do not understand: Why an RFC does not come with an executable test suite? I am pretty sure every reader of an RFC will try to create a test suite on its own.

I assume xCal and xCard formats are not yet very popular. Consequently, few people read the RFC and tried to write an implementation. This is my guess. However, it does not avoid the fact an executable test suite should (must?) be provided.

How did I find them?

This is how I found these errors. I wrote a test suite for xCal and xCard in sabre/vobject. I would love to write a test suite agnostic of the implementation, but I ran out of time. This is basically format transformation: R:xy where R can be a reflexive operator or not (depending of the versions of iCalendar and vCard we consider).

For “simple“ errata, I found the errors by testing it manually. For errata 4247 and 4261 (with the regular expressions), I found the error by applying the algorithms presented in Generate strings based on regular expressions.

Conclusion

sabre/vobject supports xCal and xCard.