mersenneforum.org  

Go Back   mersenneforum.org > Extra Stuff > Linux

Reply
 
Thread Tools
Old 2010-12-15, 12:34   #1
xilman
Bamboozled!
 
xilman's Avatar
 
"π’‰Ίπ’ŒŒπ’‡·π’†·π’€­"
May 2003
Down not across

32·11·103 Posts
Default Where to post job ad?

Apologies in advance --- I recognize this may be very off-topic for the sub-forum and will not be offended if another mod deletes this post as spam or moves it somewhere more appropriate.

A client needs someone urgently to write a high performance block device driver for the Linux kernel. They have real money to pay to someone who can do a good job. As long as the stuff works well and as long as the client has full source code, any relevant docs and a perpetual right-to-use-and-redistribute license the author can re-distribute under any license of their choosing. That is, GPL or less or more restrictive licenses are acceptable.

Does anyone have any suggestions as to where to post a job advert which includes more details?

Of course, if anyone here is willing and capable, please get in touch!


Paul
xilman is offline   Reply With Quote
Old 2010-12-15, 13:58   #2
thread
 

E7F16 Posts
Default

Can you be more specific about what kind of block device that is?
  Reply With Quote
Old 2010-12-15, 16:39   #3
xilman
Bamboozled!
 
xilman's Avatar
 
"π’‰Ίπ’ŒŒπ’‡·π’†·π’€­"
May 2003
Down not across

32×11×103 Posts
Default

Quote:
Originally Posted by thread View Post
Can you be more specific about what kind of block device that is?
Yes, though I was specifically unspecific because I didn't know whether the advert would be acceptable here.

We have a system which requires read-only random access to a database of a few terabytes. Each access is for only a small amount of data and the truly random access patterns means that there is very little, if any, point in trying to optimize for locality of storage except, perhaps, at the level of entire disks. Each datum is a kilobyte or less, smaller than the block sizes of most storage systems these days. Bandwidth is not really an issue, what we want to maximise is the number of IO operations per second (IOPS).

The system presently runs a Linux kernel on each of two dual-proc multi-core servers which are fitted with plenty of gigabytes of RAM and enough PCI buses and SATA controllers to let us hang over a hundred SSDs on them if we wish. At the moment we are finding it hard to get more than 300k IOPS no matter how much hardware we throw at the problem. The limitation is the SATA driver which itself lives under a SCSI layer. The data lives in a standard Linux-supported file system and we've pulled all the well-known optimization tricks, such as aligning the filesystem to the SSD data block boundaries, using the noop scheduler, and so forth. Note that we would be just as happy with raw device IO as with going through the filesystem; it's just that the device driver is the limiting factor at present, not the file system or SCSI layers, so the convenience of access through the file system is essentially cost-free. The SSDs can sustain several tens of thousands of IOPS (we've measured 75k IOPS from a single disk) so multiply that by a hundred disks working flat out and the theoretical peak performance should be well into the several million IOPS range.

One million IOPS is the minimum target performance. Three million would be much more in line with what we'd like and any more would be an undoubted bonus.

Why, are you capable and interested in writing a driver, or know someone who is?


Paul

Last fiddled with by xilman on 2010-12-15 at 16:40 Reason: Fix clumsy phrasing
xilman is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
what should I post ? science_man_88 science_man_88 24 2018-10-19 23:00
Post 200000 Raman Lounge 19 2016-10-07 06:17
Post numbers - what now? henryzz Forum Feedback 26 2008-12-24 14:21
Can't post to other forums Unregistered Forum Feedback 27 2007-04-04 04:56
Something that I just had to post/buy dave_0273 Lounge 1 2005-02-27 18:36

All times are UTC. The time now is 02:00.

Mon Nov 30 02:00:04 UTC 2020 up 80 days, 23:11, 3 users, load averages: 1.34, 1.24, 1.19

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.