maria-discuss team mailing list archive

[Dhananjoy Das <DDas@xxxxxxxxxxxx>]

+ Nich Piggin, Torben

Hi Daniel,
	This is Dhananjoy Das, I can be blamed for the NVM Compression,
co-authored with MariaDB :).

To help answer for the status of NVMFS, I can confirm NVMFS is a fully
POSIX compliant filesystem and currently is supported for EA (early
access) program. NVMFS is actively being developed for addressing
interesting workloads. Feel free to send out any queries you may have on
NVMFS to Nick/Torben (cc-ed).
Nick, BTW is the author of the NVMFS filesystem.

Opennvm is an independent project/effort independent of NVMFS, sorry if
any of the blogs has created any confusion. If you need details
specifically on opennvm send it my way and I should be able to find an
appropriate contact in fusion-io to help answer.

NVM-Compression uses only POSIX interfaces for its functionality
(fallocate for preallocate/TRIMs), and the IOCTL
(DFS_IOCTL_ATOMIC_WRITE_SET _IOW) is only used to enable transparent
ATOMIC-writes (invoked via io_submit()) on the specified file.

Hope the information help, please feel free to reach out directly.

Thanks,
Dhananjoy Das (Joy)



On 6/17/14, 3:04 PM, "Daniel Black" <daniel.black@xxxxxxxxxxxxx> wrote:

>
>
>----- Original Message -----
>> Hi Mark,
>> 
>> On 14 Jun 2014, at 00:44, MARK CALLAGHAN <mdcallag@xxxxxxxxx> wrote:
>> 
>> > Where can directFS be downloaded? This is open source, right?
>> >
>> 
>> From what I gather, there is a public API to this called OpenNVM
>> (http://opennvm.github.io/). But the short answer is that
>> directfs/nvmfs isn't opensource - you only get this if you have a
>> fusion-io card
>
>Further more after talking to a fusionio engineer directfs development
>has stopped 2 years ago and is unsupported by them.
>
>opennvm was mentioned as its replacement though I've got him digging up
>some closed source stuff too. Lets see if I get a nvmfs.
>
>Despite the opennvm.github.io claims it looks like mariadb doesn't
>support it as it isn't a filesystem nor does it implement some ioctl, its
>a pure API that directly operates on the /dev/fio* device.
>
>Seems like some documentation is quite out of date.
>
>I'll let you know how I get on in about a week.
>
>> Fusion-io itself is a commercial product, so nvmfs/directfs is likely
>> from them only (and is probably closed source). I take it others can
>> implement it, but haven't (yet?) ?
>> 
>> MariaDB's initial implementation of it is here:
>> http://lists.askmonty.org/pipermail/commits/2013-March/004529.html
>> (and I presume this was then ported to Percona Server as well --
>> 
>>http://www.percona.com/doc/percona-server/5.6/performance/atomic_fio.html
>>)
>> 
>> It seems the magic is at:
>> #define DFS_IOCTL_ATOMIC_WRITE_SET _IOW(0x95, 2, uint)
>> 
>> I found a presentation that talks about it (nothing about the
>> filesystem itself being opensource):
>> 
>>http://www.slideshare.net/fusionio/perconalive-accelerating-mysql-in-open
>>-source-hyperscale-systems
>> 
>> And for reference some older blog posts:
>> 
>>https://blog.mariadb.org/significant-performance-boost-with-new-mariadb-p
>>age-compression-on-fusionio/
>> https://blog.mariadb.org/mariadb-introduces-atomic-writes/
>> 
>> And attached for brevity, is a short chat I had with Jan Lindström who
>> works on the fusion-io code now:
>> 
>> 11:42 < bytee> jplindst: our fusion-io work, is based on the open APIs
>> at
>> http://opennvm.github.io/ correct?
>> 11:46 < jplindst> bytee: To be totally honest, I have no idea what are
>> the API
>> at the http page you mentioned, I have used standard POSIX
>> interfaces to access FusionIO storage, thus you will need
>> FusionIO SSD card and NVMFS file system
>> 11:47 < bytee> jplindst: nvmfs file system - where does one get it ?
>> from
>> fusion-io ?
>> 11:47 < jplindst> bytee: FusionIO card is naturally commercial
>> product, nvmfs
>> to current my understanding does not work with any other
>> device
>> 11:48 < bytee> jplindst: ok, likely because others haven't implemented
>> it
>> 11:48 < jplindst> bytee: correct
>> 11:49 < jplindst> bytee: If you have the card you should have the file
>> system
>> module
>> 11:50 < bytee> jplindst: thanks
>> 11:51 < jplindst> bytee: Last time we did discuss with FusionIO the
>> actual file
>> system was not yet open source
>> 11:51 < jplindst> bytee: not sure if that has been changed now that
>> announcements are out
>> 11:51 < bytee> jplindst: doubt there is, from what i gathere
>> 
>> I hope this helps
>> 
>> cheers,
>> -colin
>> 
>> 
>> >
>> > On Tue, Jun 10, 2014 at 5:18 PM, Daniel Black
>> > <daniel.black@xxxxxxxxxxxxx> wrote:
>> >
>> > Thanks Colin. Missed that line of the doco and/or assumed it was a
>> > capability of another file-system.
>> >
>> >
>> > ----- Original Message -----
>> > > Special ioctl doesn't work unless you use directfs (you're using
>> > > ext4); see:
>> > > https://mariadb.com/kb/en/fusionio-directfs-atomic-write-support/
>> > >
>> > > On Wednesday, 11 June 2014, Daniel Black <
>> > > daniel.black@xxxxxxxxxxxxx
>> > > > wrote:
>> > >
>> > >
>> > >
>> > > Hi,
>> > >
>> > > I'm trying to get innodb_use_atomic_writes = 1 working on an ext4
>> > > on
>> > > top of LVM on a ioDrive device with a RHEL6 kernel
>> > > 2.6.32-358.6.2.el6.x86_64
>> > >
>> > > and getting the following error. Which bit(s) need to change to
>> > > get it
>> > > working?
>> > >
>> > >
>> > > 140611 7:43:46 InnoDB: Operating system error number 25 in a file
>> > > operation.
>> > > InnoDB: Error number 25 means 'Inappropriate ioctl for device'.
>> > > InnoDB: Some operating system error numbers are described at
>> > > InnoDB:
>> > > 
>>http://dev.mysql.com/doc/refman/5.5/en/operating-system-error-codes.html
>> > > InnoDB: File name ./ibdata1
>> > > InnoDB: File operation call: 'ioctl'.
>> > > InnoDB: Error in opening ./ibdata1
>> > >
>> > > --
>> > > --
>> > > Daniel Black, Engineer @ Open Query ( http://openquery.com.au )
>> > > Remote expertise & maintenance for MySQL/MariaDB server
>> > > environments.
>> > >
>> > > _______________________________________________
>> > > Mailing list: https://launchpad.net/~maria-discuss
>> > > Post to : maria-discuss@xxxxxxxxxxxxxxxxxxx
>> > > Unsubscribe : https://launchpad.net/~maria-discuss
>> > > More help : https://help.launchpad.net/ListHelp
>> >
>> > --
>> > --
>> > Daniel Black, Engineer @ Open Query (http://openquery.com.au)
>> > Remote expertise & maintenance for MySQL/MariaDB server
>> > environments.
>> >
>> > _______________________________________________
>> > Mailing list: https://launchpad.net/~maria-discuss
>> > Post to : maria-discuss@xxxxxxxxxxxxxxxxxxx
>> > Unsubscribe : https://launchpad.net/~maria-discuss
>> > More help : https://help.launchpad.net/ListHelp
>> >
>> >
>> >
>> > --
>> > Mark Callaghan
>> > mdcallag@xxxxxxxxx
>> 
>> --
>> Colin Charles, Chief Evangelist, SkySQL - The MariaDB Company
>> blog: http://bytebot.net/blog/| t: +6-012-204-3201 | Skype:
>> colincharles
>
>-- 
>-- 
>Daniel Black, Engineer @ Open Query (http://openquery.com.au)
>Remote expertise & maintenance for MySQL/MariaDB server environments.


This e-mail (and any attachments) is confidential and may be privileged.  Any unauthorized use, copying, disclosure or dissemination of this communication is prohibited.  If you are not the intended recipient,  please notify the sender immediately and delete all copies of the message and its attachments.