Hadoop 0.20 and refactoring of the Yahoo sample code on reversed index

There are several MapReduce snippets to test and learn about Hadoop.
One of these samples is the reversed index, i.e. for each word we want to know which file it comes from. Thus the ouptut file should look like this:
hello                                     test.txt
formation formation.txt        test.txt

This example is mentioned on the Yahoo developer network, but it doesn’t work as is on version 0.20 of Hadoop.
We decided to rewrite parts of the code in order to make it compatible. This is what you will find in this blog article.

Continue reading

Integrating Solr with SPIP

Note: French version available at the second half of this blog entry.

Note Note: don’t hesitate to test our new open source package solution Datafari, which combines Apache ManifoldCF, Apache Solr and AjaxFranceLabs 🙂

SPIP is a well known open source platform. We wanted to share with you how to integrate graphically a Solr server with a SPIP server. The scenario is the following: you already have SPIP based web site, and you want to have a nice search functionnality based on the lastet Solr, to benefit from all its cool functionalities. You have set up a Solr, you have crawled your SPIP content, but now you want to have your Solr search in your SPIP website. This is what we present in this tutorial.

Continue reading

Constellio dev environment

Disclaimer: This blog is not really new, as it’s just the migration of the technical content of our website – see further down for the French version.

This tutorial explains how to start Constellio in a development environment. The first part shows how to download, setup and start Constellio in Eclipse with the default database (Derby). The second part shows how to install MySQL and to configure Constellio to use this database. Continue reading

Backup Constellio Collections and Connectors

Disclaimer: This blog is not really new, as it’s just the migration of the technical content of our website – see further down for the French version.

English: This tutorial explains how to backup and restore Collections and Connectors in Constellio.

French: Cette vidéo explique comment sauvegarder puis restaurer des Collections et des Connecteurs dans Constellio. Continue reading

Create a plugin for Constellio

Disclaimer: This blog is not really new, as it’s just the migration of the technical content of our website – see further down for the French version.

France Labs, the european partner of Doculibre on the Constellio solution, gives you this video explaining how to create a plugin for Constellio. Constellio is currently the most complete open source enterprise search solution available. Continue reading

Active Directory

Disclaimer: This blog is not really new, as it’s just the migration of the technical content of our website – see further down for the French version.

NOTE: If you are interested in using AD with Solr, you may want to look at our Datafari software (still in Alpha version), which combines Apache ManifoldCF with Solr, so it eases this kind of integration. The code is available on google code: http://code.google.com/p/datafari/

In enterprise environments, enterprise search often needs a security aspect which is not necessary for standard web search. In order to assist you, we release here a small code in order to allow Constellio 1.2 (and probably 1.3 although we didn’t test it) to connect to an Active Directory in order to do the credentials check at authentication time. Here is how it works: Continue reading

Virtual Image for Constellio 1.3

Disclaimer: This blog is not really new, as it’s just the migration of the technical content of our website – see further down for the French version.

Open source is like that, you want to give it a try by yourself. In order to for you to gain time, we set up a virtual box image of a running Constellio.
This way you can right start playing with the search and with the admin interface. So that it works right away, we even put a sample collection with a connector that crawls by default the France Labs website. So the first customisation we’d recommend is to edit this connector in order to point to the website you desire, for instance your local intranet in case you have one. The image is approx 500 MB big, so please be patient. Continue reading

Potential security risk if you use Solr together with an internet facing CMS

We recently stumbled upon a detailed article on a Solr attack using SSRF, by Nicolas Grégoire. To summarise: if you think you are safe because you have your Solr hidden behind another system, and that you have only a http server facing the web to make things ok, you may have problems you did not think about.

While reading this article, I was thinking about use cases related to CMS systems with users management, and which are accessible from the web. They are a good fit for such attacks. The good news is that Solr 4.6 solves this vulnerability. The bad news is that you need to do your migration quickly if you want to sleep well 😉

Tutorial on Authorizations for Manifold CF and Solr

NOTE: If you are interested in using ManifoldCF with Solr, you may want to look at our Datafari software , which combines Apache ManifoldCF with Solr, so it eases this kind of integration. The code is available on google code: https://github.com/francelabs/datafari

Manifold CF (MCF) provides a early-binding authorization mechanism for file searchs. The aim of this entry is to will describe this mechanism, and then to show you the different steps needed to configure MCF and Solr to use this fonctionnality.

MCF extracts ACLs from files at crawling-time, and injects them into Solr as specific fields for the Solr document. Continue reading