FollowSymLinks problems with Plesk and Strato Web Hosting: Symfony2 - symbolic links for assets

Recently I had a really hard time to figure out why I got a 403 response code when trying to access the assets I created in my symfony2 project. Using the --symlink parameter while running the assets:install command, symlinks are created to the Resources/public directories of all bundles. The assets should be accessible from the outside. The error log revealed the following:
My projects are located on a Strato server and I'm running Plesk 12 on it to manage the websites. In the past I already had trouble with different configurations overwritten or managed by Plesk, so my first guess was to check the website settings in the Plesk admin panel.

Following the Parallels KB article 27 I added the FollowSymLinks option to my projects web directory:
I tried different combinations of options and directories but none of them worked. One of the other options was -SymLinksIfOwnerMatch.

The config file which is actually edited by entering the additional HTTP server directives in Plesk is located in /var/www/vhosts/system/mydomain.tld/conf/vhost.conf. The next step was to verify that my customised vhost.conf file was actually loaded by entering invalid directives and running the apache config check. Still no success.

I made sure that the symlinks and the target directories are readable and executable. Using the following code I checked which user the webserver is running as:
I made sure that the user has read and exec rights on all of the paths and files. The error message in the log was still there. Analyzing it again I could state that the link target is for sure accessible and that symbolic links are definitely allowed!

Finally I checked the apache envvars configuration file in /etc/apache2/envvars and noticed that the webserver user and group are both www-data. Then I remembered that the current apache user is depending on the hosting account of Plesk, because PHP is running as a (Fast)CGI application. Maybe apache is trying to follow the symlinks as www-data user...
My project files all belong to the respective hosting account user. The group is always psacln which is a special Plesk group. Adding the psacln group to the www-data user solved the problem!

Symfony2 - Customize output of expanded form choices

I'm updating one of my projects to the current master of the symfony2 framework. One of the backwards incompatible changes I noticed are changes in the form framework. In one of my templates I need to customize the output of a list of checkboxes as shown in the screenshot of my data table.

The data for the checkboxes is taken from the database, so I'm using the 'entity' type in my form:

public function buildForm(FormBuilderInterface $builder, array $options)
{
    $builder
         ->add('cities', 'entity', array(
            'class' => 'MyCityBundle:City',
            'query_builder' => $this->queryBuilder,
            'required' => false,
            'expanded' => true,
            'multiple' => true,
        ))
    ;
} 

Using twig you could render all of the checkboxes without labels or anything else very easily: But obviously you don't want to render all checkboxes all at once, but in a loop. Each loop renders a table row and the checkbox is the content of one of the columns. You can loop over each of the choices with: That's fine. But what about the other data? The other data is part of the choice list which is saved in the form element and automatically retrieved from the database. The checkbox label and value can be retrieved via {{ child.vars.value }} and {{ child.vars.label }}. But there doesn't seem to be any possibility to retrieve all of the entity data in the twig template. What we need to do is saving the entity data in a separate variable and access it in the twig template. You could use the query builder, which was passed to the form element during creation to retrieve your entity data, but that would mean, that the whole data has to be retrieved again from the database, although it's already available somewhere in the form element. The way I extracted the entity data from the form element before Symfony 2.1 was: ...which doesn't work anymore. The new way (Symfony 2.1) to do it is: The only thing left is to connect the entity to the currently drawn checkbox. This can be done via the checkbox value, as this is the entity's ID attribute: This way you have full flexibility when rendering a complex expanded entity choice type.

sf2 input type date - disable native datepicker

Newer versions of the Google Chrome browser (and perhaps Firefox will follow) implement their own datepicker on input fields with type="date". This seems to be some new fancy HTML5 feature, but correct me if I'm wrong. If you are using jQuery datepicker throughout your whole project for example, you suddenly have two datepickers on that input field.

There is no possibility to turn off the native datepicker with additional attributes. You are left with three solutions to your problem:

1. Use modernizr to detect this feature and only enable jQuery datepickers, if not present:
2. Prevent standard event handling on this kind of input types: which I don't think is a good solution!
3. Render all form fields of type date with type="text" in symfony2. To enable this globally, register a global form theme in your config.yml as described in the Symfony2 documentation. My form theme consists of the following Twig code: This overwrites form_widget_simple and checks for the registered field type before setting it for the input field. If it finds 'date' it is overwritten with 'text'. Very nice and quick solution in my opinion, hope that helps you.

Sublime Text 2 - Snippet to create get/set methods

Sublime Text 2 is one of the most advanced editors out there, available for Windows, Mac and Linux. I'm using it on an every day basis to write my PHP applications.

Sublime Text is a sophisticated text editor for code, html and prose. You'll love the slick user interface and extraordinary features. (www.sublimetext.com)

Its snippet system allows you to create snippets that are triggered by a sequence of characters you can define, followed by tab key. The syntax is really easy. To create a snippet choose "Tools, New Snippet".

The snippet I'd like to show you today is a huge timesaver. I use it excessively in my form models in symfony2. After declaring a model variable as private or protected I need to create a get and set method for it. Copy paste and substitute the variable name was my choice until today. The following snippet allows you to use the following workflow:

1. type 'getset'
2. press the tab key
3. type your variable name
4. done.

The result:

Many to many search with doctrine2 and symfony2

My last post was about how to search for a many to one entity where I needed to check the values of two attributes by using "exists". I couldn't find any other "clean" ORM/DQL query to do this.

While experimenting with many to many relations I encountered how to easily retrieve all users that are associated with a certain work category. Many to many relations in doctrine create a relational table between two entities, which is no entity itself. You cannot use the table to create joins. I'd like to cite the doctrine2 orm reference:

"Why are many-to-many associations less common? Because frequently you want to associate additional attributes with an association, in which case you introduce an association class. Consequently, the direct many-to-many association disappears and is replaced by one-to-many/many-to-one associations between the 3 participating classes."

The association class is your third entity which you can use to create your search queries using JOINs for example.

In this case we have a pure many to many relationship. I built a form with an element of type entity which represents the work categories.

As this draws a multiselect control with the ID of the work categories as value on submitting the form, you have access to them in your search query. The key is to use "MEMBER OF" then:


Surprisingly for me, this creates a very similar query as I used for the one-to-many search query:

AND EXISTS (SELECT 1 FROM users_workcategories u2_ INNER JOIN user_work_category u3_ ON u2_.workcategory_id = u3_.id WHERE u2_.user_id = u0_.id AND u3_.id = ?) AND EXISTS (SELECT 1 FROM promoters_workcategories u2_ INNER JOIN user_work_category u3_ ON u2_.workcategory_id = u3_.id WHERE u2_.user_id = u0_.id AND u3_.id = ?)

Doctrine is aware of of the third relational table and uses it for JOINs to assemble the native SQL query that looks far more complicated than the DQL string. For people not familiar with advanced SQL queries, this is a live safer. On the other hand, SQL people surely need to be argued into this new approach.

Search form using symfony2 and doctrine2

This post is about creating a search form and using the doctrine2 query builder to dynamic built the query. You need an object you can bind the form values to. I used the symfony2 entity generator to build my heavy search form data holder really quickly. Let it create an empty repository class for you, which will hold the search function later. Use the symfony2 form generator to create a form based on the entity. As this entity won't be saved in the database the doctrine annotations are not needed. I kept them for later use and just removed the "Entity" annotation.

Next thing you do is writing a template for your form and to specify some validation rules. My controller action which receives the search POST looks like this:



We need to implement the findBySearchCriteria function in the empty repository class of the user entity. This function returns a query builder object which we can do everything we want with.

The following examples show what is possible with the doctrine2 query builder in the search function.

Many-to-one search

The user has a languages attribute which is of type Language. The entity Language has three attributes:

1. a language name (e.g. english, french, etc)
2. points that define how good the language is spoken by the user (10 very good, 1 very bad)
3. a back reference to the user

We want to find users that marked their english knowledge at least with 5 points and their french with at least 7 points. To accomplish this, we need to check, if the user has an language entry which fulfills the given criteria. We check, if such a language entry exists.

As "exists" in the query builder expects a dql subquery, we give it one.

"u" is the alias for the User class of the main query builder. The same method is used for the french language points.

The resulting DQL is:

SELECT u, l FROM Acme\UserBundle\Entity\User u WHERE (EXISTS( SELECT langE FROM AcmeUserBundle:Language langE WHERE langE.user = u AND langE.language = :langName AND langE.points BETWEEN 10 AND :points))

Nested AND / OR queries

Next requirement is that the zip code of the user entity has to lie in a given range of zip codes. The user who defines the search criteria is able to provide a list of zip code ranges. The format for a range is "xxxxx-xxxxx", so a zip code consists of five digits.

The zip code range search query needs to be built using OR. The user's zip code can lie in one range OR another. Native SQL could look like this:

WHERE (zipCode BETWEEN :zipCodeFrom1 AND :zipCodeTo1) OR (zipCode BETWEEN :zipCodeFrom2 AND :zipCodeTo2)

To combine this with the language query, the OR part has to be nested within an AND part:

WHERE ((zipCode BETWEEN :zipCodeFrom1 AND :zipCodeTo1) OR (zipCode BETWEEN :zipCodeFrom2 AND :zipCodeTo2)) AND (EXISTS( SELECT .... 

To accomplish the OR nesting you can use add() on the orx expression. The following example shows how to do that:


This results in a really nice nested query like this:

SELECT u FROM Acme\UserBundle\Entity\User u WHERE ((u.zipCode BETWEEN :zipFrom1 AND :zipTo1) OR (u.zipCode BETWEEN :zipFrom2 AND :zipTo2)) AND (EXISTS( SELECT langE FROM AcmeUserBundle:Language langE WHERE langE.user = u AND langE.language = :langName AND langE.points BETWEEN 10 AND :points))

Building nested queries is really easy. It took me some time to find out that I can use add() on the orx expression to dynamically add parts to it.

Notes on Zend SOAP webservice development

Developing SOAP webservices with the Zend Framework is very easy, but there are some things that aren't explicitly mentioned in the documentation. Hopefully this hints will help you to save time.

WSDL caching

While developing a webservice application do not forget to disable WSDL caching, e.g. in your php.ini configuration file.

soap.wsdl_cache_enabled=0

Set path to SOAP server in WSDL file

When you use a controller for your webservice and desire to have two actions, one for the automatically generated WSDL file and one for the server, use the setUri method of the Zend_Soap_AutoDiscover class to set the URI of the server.

$wsdl->setUri('http://example.localhost/index/soapserver');

Error handling

Error handling with exceptions is easy. I suggest to create your own Exception class first, it can be empty for now:

In your webservice class throw your new exception on invalid input for example.
On SOAP server side, you have to register your Exception class:


On Soap client side embed your request in a try - catch block:

Which for example outputs: 'SOAP ERROR: [Receiver] The product ID does not exist!'

The code

The webservice controller:


The webservice class:


The custom Exception class:

Google Chrome double request session problem

While working on an older PHP project based on a self made framework I noticed strange behaviour when running the project in Chrome. Everything worked fine in all other browsers, no matter, if it was production or development environment.

Each time I logged in the session ID cookie changed. The consequence was, that when trying to reload the page or open other restricted pages the system logged me out.

Even though everthing worked fine on other browsers, I tried to find the bug in the PHP code. I thought I missed something, set wrong headers which Chrome interpreted more strictly than other browsers or something similar. None of that was true. I debugged the code responsible for the session, made sure nothing is going to fall through the cracks. Code that was intended to run once on page execution ran twice, which lead me to the idea, that it's a forwarding/reloading issue.

I checked the $_SERVER array for header information, which differed between browsers. But there was nothing suspicious. REQUEST_URI then revealed the problem: While you would expect only one request to '/' Chrome requested '/favicon.ico' in a second request. This somehow caused a reload with a seperated cookie space, because the reloaded page couldn't read the session cookie and created a new one. I'm not sure what happened in detail but providing a favicon.ico file helped.

I'd like to emphasize that I didn't use a favicon in the HTML, perhaps there is a flag set in the server configuration which is ignored by other browsers.

Hope this helps someone, this took me some time to figure out.

Font problems with wkhtmltopdf

Recently I ran into problems with font kerning and font sizes when creating PDFs with wkhtmltopdf on different machines. I create PDFs on my ubuntu machine locally and on a FreeBSD server remotely.

Unfortunately font sizes differed a lot and from the beginning the kerning wasn't as I expected. The space after characters like 'v' is too small or too big, depending on the chosen font size. Sometimes it happens that two characters get rendered  one on the other. This is not acceptable.

I found out that this topic is already discussed in the wkhtml bugtracker. Users have different approaches to that problem, but none of the solutions is acceptable for me. Running an XServer is not possible and running xvfb as a wrapper to the tool sounds a lot slower than using wkhtmltopdf 'natively' and certainly is!

After experimenting with CSS modifications concerning font sizes, letter spacing, --zoom and --disable-smart-shrinking / --enable-smart-shrinking parameters to wkhtmltopdf I finally found a working solution. It's very easy to apply and has no disadvantages.

I need polish characters in my texts, so I downloaded high quality fonts from TeX Gyre Project. They provide their fonts in ODF format instead of the commonly used TTF format. I don't know, if the fonts of the Gyre Project solve the problem or if it is the newer ODF format that eliminates certain limitations of the TTF format when used on different operating systems.

In your external or internal stylesheet of your HTML file you'd like to convert, define a custom font face like this:


Don't forget to use the correct path to your font. Whenever you want to use it, use your new defined font family:


This eliminates all problems arising due to different font versions on different systems and the kerning bug in wkhtmltopdf version 0.10.0_beta5

Filling PDF forms using XFDF

Description

This article is about user data that can be provided in PDF form back to the user. Don't use old methods in the web which suggest to use FDF. FDF and the PHP FDF module are deprecated, don't support UTF-8 and you'll have only problems with it ;). 

One common use case is a web form that requires the user to input information for an application that has to be sent by snail mail for confirmation or to print the information in a user friendly form that is exchangeable.

The key is creating an XFDF file that holds all the user information connected to IDs of form input fields that are defined in a PDF file with empty inputs. The empty fields are then filled with the user information. The URL to the PDF file is defined in the XFDF file. If the user opens the XFDF file with the help of the Adobe Reader browser plugin, all happens in the user's browser window. An alternative for the user is downloading the XFDF file to open it later.

Advantages:

1. As filled PDF forms can't be saved (but can be printed), this is an alternative to keep the inputted data,
2. you don't need HTML to PDF converters. Create professional looking PDFs in any program you like

Disadvantages:

1. If the PDF file is available online, the user has to have an active internet connection whenever he or she opens the XFDF file, 
2. Adobe Reader asks, if it should fill out the PDF with possibly unsecure data. The PDF stays empty without user's confirmation.

One workaround for the 1st disadvantage is PDFToolkit. It allows to flatten a PDF file with the XFDF file to create a user ready downloadable PDF file, but the data is not editable anymore. The 2nd workaround is to download the empty PDF file to disc and change the URL in the XFDF file which is in XML format.

Creating PDF forms with OpenOffice

OpenOffice has a toolbar with form elements. You can find it in the View->Toolbars->Form Elements menu. Just place the elements you need in the document and export the whole thing as PDF. Don't forget  to check the "Create PDF Form" option. Please be aware of the IDs of the form elements in the document. You can set the ID by chosing "Properties" in the context menu of the element. To check the form elements afterwards you can use PDFToolkit:

pdftk test.pdf dump_data_fields
\--\-
FieldType: Text
FieldName: textelement1

FieldNameAlt:
FieldFlags: 0
FieldValue:
FieldValueDefault:
FieldJustification: Left

Generating the XFDF file

The specification I used can be found on Adobe Website. This is the PHP code to create an XFDF file out of an array of user input data fields.


And this is how to use it:


As you can see, only the XFDF file is outputted. The Adobe Reader plugin parses it and tries to download the corresponding PDF file to fill in the user data.

Dynamic HTML to PDF conversion

There are a lot of libraries out there which allow you to create PDFs from HTML documents. Some of them even allow to use external stylesheets, others allow inline styles only. They have one disadvantage in common: the PDF never looks like your original HTML file. Other limitations include i.e.:

• No support for floating,
• no support for ordered lists,
• no support for nested tables,
• in general table support is weak for all libraries I tested,
• only a subset of CSS styles is supported,
• dynamic content placement is very difficult,
• problems on page breaks,
• etc.

The easiest way is to just print the HTML page to a PDF file. The generated files are nearly 100% as you expect them to be. The best solution I found to accomplish this is wkhtmltopdf. Ask your webhosting provider to install this tool, if you don't have the possiblity to install it by yourself. I heard of people that managed it to install wkhtmltopdf in their home directory successfully.

wkhtmltopdf uses QT libraries. These libraries include the Webkit rendering engine for Apple Safari Browsers. So you can expect to have all CSS and HTML features working that Safari currently supports. You have to compile the author's patched QT libraries to have full functionality without the need of using a running XServer. Alternatively use the static executables.

You can find a PHP Class you can use with wkhtmltopdf. It's pretty self explaining and the example application should help you creating your first PDF file. I create PDF files by rendering an HTML file out of my PHP application, saving it to a writable directory with other necessary external files like images, js and css and executing the wkhtmltopdf converter.

It's a great tool and creates small PDF files. I'm missing security features like password protection, though.