Home Archive RSS Login

I learned something new today

MongoDB - GridFS

Intro from the MongoDB website:

GridFS is a storage specification for large objects in MongoDB. It works by splitting large object into small chunks, usually 256k in size. Each chunk is stored as a separate document in a chunks collection. Metadata about the file, including the filename, content type, and any optional information needed by the developer, is stored as a document in a files collection.

So for any given file stored using GridFS, there will exist one document in files collection and one or more documents in the chunks collection.


GridFS links from the MongoDB website:

http://www.mongodb.org/display/DOCS/GridFS
http://www.mongodb.org/display/DOCS/GridFS+Specification

PHP MongoGridGS Class
http://php.net/manual/en/class.mongogridfs.php

Example:
http://www.lightcubesolutions.com/blog/?p=209

I tested this out in PHP and following are some sample codes that may help.


1. PHP code to store uploaded file into the MongoDB GridFS.

if (isset($_POST["PHPSESSID"])) {
session_id($_POST["PHPSESSID"]);
}
session_start();

$conn = new Mongo("127.0.0.1:27017"); // Connect
$db = $conn->uniform_server; // Select DB
$db->authenticate("user","password"); // Authenticate to MongoDB
$grid = $db->getGridFS(); // Initialize GridFS
$name = $_FILES['Filedata']['name']; // Get Uploaded file name
$type = $_FILES['Filedata']['type']; // Try to get file extension
$id = $grid->storeUpload('Filedata',$name); // Store uploaded file to GridFS

/* Mime types did not work properly on my setup, hence the following logic */
$ext = substr($name,-3);
switch($ext) {
case "jpg": $type = "image/jpeg"; break;
case "gif": $type = "image/gif"; break;
case "png": $type = "image/png"; break;
case "txt": $type = "text/plain"; break;
case "pdf": $type = "application/pdf"; break;
case "zip": $type = "application/x-zip"; break;
}

/* Add additional metadata related to the file if required */
$files = $db->fs->files;
$files->update(array("filename" => $name), array('$set' => array("contentType" => $type, "aliases" => null, "metadata" => null)));

$conn->close(); // Close connection

exit(0);
?>


2. PHP code to list all the files in the GridFS (in a particular database)

$conn = new Mongo("127.0.0.1:27017"); // Connect
$db = $conn->uniform_server; // Select DB
$db->authenticate("user","password"); // Authenticate to MongoDB
$grid = $db->getGridFS(); // Initialize GridFS

$cursor = $grid->find();

foreach ($cursor as $obj) { // iterate through the results
echo 'Filename: '.$obj->getFilename().' Size: '.$obj->getSize().'
';
}

$conn->close(); // Disconnect from Server
exit(0);
?>


3. PHP code to download files stored in GridFS

$conn = new Mongo("127.0.0.1:27017"); // Connect
$db = $conn->uniform_server; // Select DB
$db->authenticate("user","password"); // Authenticate to MongoDB
$grid = $db->getGridFS(); // Initialize GridFS

$ask = $_REQUEST['file']; // Get filename requested

$file = $grid->findOne(array('filename' => $ask));
$files = $db->fs->files;
$file1 = $files->findOne(array('filename' => $ask));
$id = $file->file['_id'];

if ( (substr($ask,-3) == 'zip') || (substr($ask,-3) == 'pdf') ) {
/* Any file types you want to be downloaded can be listed in this */
header('Content-Type: application/octet-stream');
header('Content-Disposition: attachment; filename='.$ask);
header('Content-Transfer-Encoding: binary');
$cursor = $db->fs->chunks->find(array("files_id" => $id))->sort(array("n" => 1));
foreach($cursor as $chunk) {
echo $chunk['data']->bin;
}
}
else {
header('Content-Type: '.$file1["contentType"]);
echo $file->getBytes();
}

$conn->close(); // Disconnect from Server
exit(0);
?>


4. PHP code to delete a file from GridFS

$conn = new Mongo("127.0.0.1:27017"); // Connect
$db = $conn->uniform_server; // Select DB
$db->authenticate("user","password"); // Authenticate to MongoDB
$grid = $db->getGridFS(); // Initialize GridFS

$filename = $_REQUEST["file"]; // Get requested filename

$file = $grid->findOne($filename); // Find file in GridFS
$id = $file->file['_id']; // Get the files ID
$grid->delete($id); // Delete the file

$conn->close(); // Disconnect from Server
exit(0);
?>

Author: Prit -  Date: 22 Jul 2010 12:13
Tags: mongodb,programming -  Visits: 55131 -  No Comments




MongoDB - Replica Pair

Intro from the MongoDB website:

Mongo supports a concept of replica pairs. These databases automatically coordinate which is the master and which is the slave at a given point in time.

At startup, the databases will negotiate which is master and which is slave. Upon an outage of one database server, the other will automatically take over and become master from that point on. In the event of another failure in the future, master status would transfer back to the other server. The databases manage this themselves internally.


You can read about the replica pair from the MongoDB website at the below link:
http://www.mongodb.org/display/DOCS/Replica+Pairs

Following is an example of how to run a replica pair. A replica pair is different from the master-slave configuration. In a master slave config, the slave is read-only and acts like a backup database. In a replica pair, if any of the databases go down, then the other takes over as master.
./mongod --pairwith localhost:27018 --arbiter localhost:27019 --dbpath ~/data1
./mongod --port 27018 --pairwith localhost:27017 --arbiter localhost:27019 --dbpath ~/data2
./mongod --port 27019 #arbiter

The arbiter is a mongodb server that runs on a third machine that helps negitiate between the paired servers to determine which one should be master.

Following is the PHP example code to connect when using a replica pair:
$m = new Mongo("mongodb://localhost:27017,localhost:27018");

I tested this by stopping one server at a time and the other took over as master and the application continued to work without any failures or unavailability.
Author: Prit -  Date: 22 Jul 2010 12:07
Tags: mongodb,programming -  Visits: 5104 -  No Comments




MongoDB - Master Slave Setup

You can read more about the master-slave configuration at the MongoDB website:
http://www.mongodb.org/display/DOCS/Master+Slave

Following commands to start a master slave setup in Ubuntu and monitor outputs if the logs are the same.
Start the master:
sudo /usr/bin/mongod --master --rest --config /etc/mongodb.conf

Start the slave:
sudo /usr/bin/mongod --rest --slave --source 127.0.0.1:27017 --dbpath /data/slavedb/ --fastsync --autoresync --config /etc/mongodb2.conf

Monitor logs from command line:
tail -f /var/log/mongodb/mongodb.log

Also, when I had authentication issues, the following webpage helped resolve:
http://myadventuresincoding.wordpress.com/2010/04/26/mongodb-connecting-a-slave-to-a-master-running-in-auth-mode/

In auth mode, for replication to work, you need to add the following kind of similar users to the "local" db of both the master and the slave.
db.addUser("repl","replpassword")

Author: Prit -  Date: 22 Jul 2010 12:06
Tags: mongodb,programming -  Visits: 6160 -  No Comments




MongoDB - Authentication

You can read more about authentication from the MongoDB website:
http://www.mongodb.org/display/DOCS/Security+and+Authentication

To do authentication, you need to start the mongod process using the --auth command line

Continue reading
Author: Prit -  Date: 22 Jul 2010 12:05
Tags: mongodb,programming -  Visits: 4834 -  No Comments




MongoDB

Recently I have been reading about the new breed of databases that are called "document oriented" databases. One of them that seems to get a lot of attention is MongoDB. You can read more about it from their website:
http://www.mongodb.org/

To test it out, you can create Mongo databases in http://mongohq.com.

Uniform Server (portable) in Windows

If you are interested in playing with MongoDB in Windows, you can use a portable WAMP server called Uniform Server with a Mongo Plugin

Continue reading
Author: Prit -  Date: 22 Jul 2010 10:13
Tags: mongodb,programming -  Visits: 5203 -  No Comments




Pages: ... [3] [4] [5] [6] [7] [8] ...