Debian Bug report logs - #810822
RFP: MooseFS -- fault tolerant, highly available, highly performing, scaling-out, network distributed file system

Reply or subscribe to this bug.

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Piotr Robert Konopelko <piotr.konopelko@moosefs.com>

To: Debian Bug Tracking System <submit@bugs.debian.org>, debian-mentors@lists.debian.org

Cc: MooseFS Technical Support Department <dwt@moosefs.com>

Subject: ITP: MooseFS

Date: Tue, 12 Jan 2016 16:27:06 +0100

[Message part 1 (text/plain, inline)]

Subject: ITP: MooseFS -- MooseFS (MFS) is a fault tolerant, highly available, highly performing, scaling-out, network distributed file system. It spreads data over several physical servers which are visible to the user as one resource. For standard file operations MooseFS mounted with FUSE acts as other Unix-alike file systems.
X-Debbugs-Cc: piotr.konopelko@moosefs.com, debian-devel@lists.debian.org, debian-mentors@lists.debian.org, dwt@moosefs.com
Package: wnpp
Owner: Piotr Robert Konopelko <piotr.konopelko@moosefs.com>
Severity: wishlist

  Package name		: moosefs-master, moosefs-metalogger, moosefs-chunkserver, moosefs-client, moosefs-cgi, moosefs-cgiserv, moosefs-netdump
  Version				: 2.0.83-1
  Upstream Author		: Jakub Kruszona-Zawadzki <jakub.kruszona@moosefs.com <mailto:jakub.kruszona@moosefs.com>>
  URL				: https://moosefs.com <https://moosefs.com/>
  Sources URL			: https://moosefs.com/download/sources.html <https://moosefs.com/download/sources.html>
  License				: GPLv2
  Programming Lang	: C, Python
  Description			: MooseFS (MFS) is a fault tolerant, highly available, highly performing, scaling-out, network distributed file system. It spreads data over several physical servers which are visible to the user as one resource. For standard file operations MooseFS mounted with FUSE acts as other Unix-alike file systems.

  Dependencies		: libpcap0.8, python, libfuse2

  Long description:

    MooseFS (MFS) is a fault tolerant, highly available, highly performing, scaling-out, network distributed file system. It spreads data over several physical servers which are visible to the user as one resource. For standard file operations MooseFS mounted with FUSE acts as other Unix-alike file systems:

      * A hierarchical structure (directory tree)
      * Stores POSIX file attributes (permissions, last access and modification times)
      * Supports special files (block and character devices, pipes and sockets)
      * Symbolic links (file names pointing to target files, not necessarily on MooseFS) and hard links (different names of files which refer to the same data on MooseFS)
      * Access to the file system can be limited based on IP address and/or password

    Distinctive features of MooseFS are:

      * High availability
      * High reliability (several copies of the data can be stored on separate computers)
      * Capacity is dynamically expandable by simply adding new computers/disks
      * Deleted files are retained for a configurable period of time (a file system level "trash bin")
      * Coherent snapshots of files, even while the file is being written/accessed


We're already publishing own packages repository (https://moosefs.com/download/ubuntudebian.html <https://moosefs.com/download/ubuntudebian.html>).
We would like to make MooseFS available directly in Debian! :)


Best regards,

-- 
Piotr Robert Konopelko
MooseFS Technical Support Engineer | moosefs.com <https://moosefs.com/>

[Message part 2 (text/html, inline)]

Message #14 received at 810822@bugs.debian.org (full text, mbox, reply):

From: Dmitry Smirnov <onlyjob@debian.org>

To: Jakub Kruszona-Zawadzki <jakub.kruszona@gemius.com>, MooseFS Technical Support Department <dwt@moosefs.com>

Cc: László Böszörményi (GCS) <gcs@debian.org>, Piotr Robert Konopelko <piotr.konopelko@moosefs.com>, 810822@bugs.debian.org

Subject: Re: Debian repository contains a fork of MooseFS

Date: Sat, 16 Jan 2016 01:04:35 +1100

[Message part 1 (text/plain, inline)]

On Thu, 14 Jan 2016 07:01:52 AM Jakub Kruszona-Zawadzki wrote:
> We just want to add MooseFS to official Debian repositories.
> [..]
> For me both packages should be available in Debian repositories,

Except for promoting MooseFS I do not see any benefits from introducing 
MooseFS to Debian but there are numerous cons.


> The only problem is that they didn't change name of
> binaries (which in my personal opinion is kind of stupid - moosefs =
> mfsmaster,mfschunkserver,mfsmount etc, LizardFS should be
> lfsmaster,lfschunkservder,lfsmount etc.).

As far as I'm aware that was deliberate choice in order to allow seamless 
upgrades from MooseFS. I think renaming is planned but it is a low priority 
thing.


> They [LizardFS] actively develop their product, but much slower than
> original (my personal opinion, but we check their product regularly).

Well thankfully it is not a race... Since MooseFS do not have public bug 
tracker or VCS I supose you might be in unique position to compare speed of 
development. However independent observer will hardly be able to reach the 
same conclusion.


> This is a fork,
> so they develop this product in different way and of course it also means
> that in some rare cases they product is even better than MooseFS. So I
> absolutely agree that for sure there are some people who will find
> LizardFS better than MooseFS and therefore both projects should be
> available in the package repositories.

I disagree. For Debian there will be undesirable overhead from having two 
similar software products. It would also cause needless confusion for our 
users. When choosing between the two I give preference to LizardFS due to 
greater potential and development transparency. At this time introducing 
MooseFS to Debian is unnecessary but situation may change in the future.


> We know everything about LizardFS. Whole company is run by the guy who was
> fired from our company (he was not a developer, but our it administrator,
> so he knew MooseFS very well). He "stole" two developers from our company
> (they never were involved in moosefs developing, but they also knew this
> product). The whole story is not very nice, but leave it. This is their
> right to make fork of actively developed project. As you wrote - they have
> their own investor and therefore their own goal. I do not care about it.

Thank you for interesting insights about projects history.

-- 
Best wishes,
 Dmitry Smirnov.

---

It is a mistake to try to look too far ahead. The chain of destiny can only
be grasped one link at a time.
        -- Winston Churchill

[signature.asc (application/pgp-signature, inline)]

Message #19 received at 810822@bugs.debian.org (full text, mbox, reply):

From: Dmitry Smirnov <onlyjob@debian.org>

To: debian-mentors@lists.debian.org, Piotr Robert Konopelko <piotr.konopelko@moosefs.com>, 810822@bugs.debian.org, MooseFS Technical Support Department <dwt@moosefs.com>

Subject: Re: Bug#810822: ITP: MooseFS

Date: Sat, 16 Jan 2016 01:09:11 +1100

[Message part 1 (text/plain, inline)]

On Tue, 12 Jan 2016 04:27:06 PM Piotr Robert Konopelko wrote:
> We're already publishing own packages repository
> (https://moosefs.com/download/ubuntudebian.html
> <https://moosefs.com/download/ubuntudebian.html>). We would like to make
> MooseFS available directly in Debian! :)

In November 2014 when I was working on introducing LizardFS (a fork of 
MooseFS) to Debian, there were no publicly available source code for MooseFS 
-- no tarballs or source code repository. Later it became possible to request 
MooseFS sources by filling web form and hoping that sources will be sent over 
email. Even though now MooseFS sources can be freely downloaded there are no 
public bug tracker and no public VCS. Lack of VCS means one can not isolate 
(and backport) fixes without great difficulties. 

Moreover only crippled "community edition" of MooseFS is free software as 
MooseFS developers apparently focused on proprietary "PRO" edition.

Debian already have MooseFS' fork -- LizardFS. To my knowledge at the moment 
MooseFS do not offer noticeable advantages over LizardFS while the latter 
seems to have slightly more features.

For quite a while LizardFS is developed with community using public VCS and 
bug tracker (GitHub) as well as Gerrit code review system and continuous 
integration system. LizardFS have more development transparency than MooseFS 
ever had.

I believe that poor governance of MooseFS motivated forking and it appears 
that MooseFS developers still did not learn their lesson.

***
Based on the above I recommend to refrain from introducing MooseFS to Debian.
***

Please note that IMHO MooseFS versus LizardFS situation have many 
similarities with MySQL vs. MariaDB situation where poor Oracle's governance 
and focus on proprietary addons discourage community from working with them.
(MySQL is not as bad as MooseFS because MySQL have public bug tracker).

As I object to introduction of MooseFS to Debian I would object to 
introduction of MySQL if MariaDB were already available.

Having both is a burden and non-negligible overhead.

With all due respect to MooseFS's former innovations and legacy I think for 
now it would be best to refrain from debianising MooseFS and re-evaluate 
situation in the future if there will be any development.

-- 
Cheers,
 Dmitry Smirnov.

---

We often refuse to accept an idea merely because the way in which it has
been expressed is unsympathetic to us.
        -- Friedrich Nietzsche

[signature.asc (application/pgp-signature, inline)]

Message #24 received at 810822@bugs.debian.org (full text, mbox, reply):

From: Paul Wise <pabs@debian.org>

To: Dmitry Smirnov <onlyjob@debian.org>

Cc: debian-mentors@lists.debian.org, Piotr Robert Konopelko <piotr.konopelko@moosefs.com>, 810822@bugs.debian.org, MooseFS Technical Support Department <dwt@moosefs.com>

Subject: Re: Bug#810822: ITP: MooseFS

Date: Sun, 17 Jan 2016 20:34:50 +0800

Radical suggestion: end MooseFS development, MooseFS devs move to
working in public within the LizardFS community, porting MooseFS
features over. Could help repair trust.

-- 
bye,
pabs

http://wiki.debian.org/PaulWise
http://bonedaddy.net/pabs3/

Message #29 received at 810822@bugs.debian.org (full text, mbox, reply):

From: Jakub Kruszona-Zawadzki <jakub.kruszona@gemius.com>

To: Dmitry Smirnov <onlyjob@debian.org>

Cc: debian-mentors@lists.debian.org, Piotr Robert Konopelko <piotr.konopelko@moosefs.com>, 810822@bugs.debian.org, MooseFS Technical Support Department <dwt@moosefs.com>

Subject: Re: Bug#810822: ITP: MooseFS

Date: Mon, 18 Jan 2016 07:12:04 +0100

On 15 Jan, 2016, at 15:09, Dmitry Smirnov <onlyjob@debian.org> wrote:

> On Tue, 12 Jan 2016 04:27:06 PM Piotr Robert Konopelko wrote:
>> We're already publishing own packages repository
>> (https://moosefs.com/download/ubuntudebian.html
>> <https://moosefs.com/download/ubuntudebian.html>). We would like to make
>> MooseFS available directly in Debian! :)
> 
> In November 2014 when I was working on introducing LizardFS (a fork of 
> MooseFS) to Debian, there were no publicly available source code for MooseFS 
> -- no tarballs or source code repository. Later it became possible to request 
> MooseFS sources by filling web form and hoping that sources will be sent over 
> email. Even though now MooseFS sources can be freely downloaded there are no 
> public bug tracker and no public VCS. Lack of VCS means one can not isolate 
> (and backport) fixes without great difficulties. 

We may make official clone of GPL'ed version on GitHub if it change anything.

> 
> Moreover only crippled "community edition" of MooseFS is free software as 
> MooseFS developers apparently focused on proprietary "PRO" edition.

This is not true. 99.9% of my time I spend on GPL'ed version. Now there is only one feature in PRO version that is not available in GPL version. We do not plan to change that (maybe in the future exchange this feature to another).

> 
> Debian already have MooseFS' fork -- LizardFS. To my knowledge at the moment 
> MooseFS do not offer noticeable advantages over LizardFS while the latter 
> seems to have slightly more features.

You are very wrong. We have users who switched from LizardFS. Stability, efficiency for example. Quite important advantages. Not for everyone, but at least for serious users.

> 
> For quite a while LizardFS is developed with community using public VCS and 
> bug tracker (GitHub) as well as Gerrit code review system and continuous 
> integration system. LizardFS have more development transparency than MooseFS 
> ever had.

And in case of file system it is not good idea. 

> 
> I believe that poor governance of MooseFS motivated forking and it appears 
> that MooseFS developers still did not learn their lesson.

No. This was financial decision.

> 
> ***
> Based on the above I recommend to refrain from introducing MooseFS to Debian.
> ***
> 
> Please note that IMHO MooseFS versus LizardFS situation have many 
> similarities with MySQL vs. MariaDB situation where poor Oracle's governance 
> and focus on proprietary addons discourage community from working with them.
> (MySQL is not as bad as MooseFS because MySQL have public bug tracker).

You are not right. MariaDB is developed by original author of MySQL, so the spirit of MySQL now is in MariaDB.

I invented MooseFS and I'm still developing MooseFS. Also in terms of money. We do not make money on MooseFS. The PRO version is just for us to help developing it at all. Let's say that thanks to pro version of MooseFS we are able to make GPL version of MooseFS. Thanks to that I'm able to work 8 hours a day developing MooseFS, so we made PRO version to actually improve developing of GPL version.

> 
> As I object to introduction of MooseFS to Debian I would object to 
> introduction of MySQL if MariaDB were already available.

Why? People should have right to choose. 

> 
> Having both is a burden and non-negligible overhead.
> 
> With all due respect to MooseFS's former innovations and legacy I think for 
> now it would be best to refrain from debianising MooseFS and re-evaluate 
> situation in the future if there will be any development.

What kind of "development" are you talking about?

> 
> -- 
> Cheers,
> Dmitry Smirnov.
> 
> ---
> 
> We often refuse to accept an idea merely because the way in which it has
> been expressed is unsympathetic to us.
>        -- Friedrich Nietzsche

-- 
Regards,
Jakub Kruszona-Zawadzki
- - - - - - - - - - - - - - - -
Segmentation fault (core dumped)
Phone: +48 602 212 039

Message #34 received at 810822@bugs.debian.org (full text, mbox, reply):

From: Jakub Kruszona-Zawadzki <jakub.kruszona@gemius.com>

To: Paul Wise <pabs@debian.org>

Cc: Dmitry Smirnov <onlyjob@debian.org>, debian-mentors@lists.debian.org, Piotr Robert Konopelko <piotr.konopelko@moosefs.com>, 810822@bugs.debian.org, MooseFS Technical Support Department <dwt@moosefs.com>

Subject: Re: Bug#810822: ITP: MooseFS

Date: Mon, 18 Jan 2016 07:38:38 +0100

On 17 Jan, 2016, at 13:34, Paul Wise <pabs@debian.org> wrote:

> Radical suggestion: end MooseFS development, MooseFS devs move to
> working in public within the LizardFS community, porting MooseFS
> features over. Could help repair trust.

Radical and ... kind of stupid. I have similar idea. end LizardFS development. LizardFS devs could help developing MooseFS. But ... hmm ... there are no interesting features to port from Lizard to Moose.

The question is why anybody forked actively developed product instead of helping in development of original? Don't you think that it is very rude? They never wanted to help us. At one point they wanted to buy us, but we disagreed, so they made fork.

By the way all LizardFS developers are on payroll, there is no "developing community" behind LizardFS - only company from USA. You people have sometimes tendency to help cheaters instead of helping honest and hardworking people.

> 
> -- 
> bye,
> pabs
> 
> http://wiki.debian.org/PaulWise
> http://bonedaddy.net/pabs3/

-- 
Pozdrawiam,
Jakub Kruszona-Zawadzki
- - - - - - - - - - - - - - - -
Segmentation fault (core dumped)
Tel. +48 602 212 039

Message #39 received at 810822@bugs.debian.org (full text, mbox, reply):

From: Jakub Kruszona-Zawadzki <jakub.kruszona@gemius.com>

To: Dmitry Smirnov <onlyjob@debian.org>

Cc: MooseFS Technical Support Department <dwt@moosefs.com>, "László Böszörményi (GCS)" <gcs@debian.org>, Piotr Robert Konopelko <piotr.konopelko@moosefs.com>, 810822@bugs.debian.org

Subject: Re: Debian repository contains a fork of MooseFS

Date: Mon, 18 Jan 2016 06:53:05 +0100

On 15 Jan, 2016, at 15:04, Dmitry Smirnov <onlyjob@debian.org> wrote:

> On Thu, 14 Jan 2016 07:01:52 AM Jakub Kruszona-Zawadzki wrote:
>> We just want to add MooseFS to official Debian repositories.
>> [..]
>> For me both packages should be available in Debian repositories,
> 
> Except for promoting MooseFS I do not see any benefits from introducing 
> MooseFS to Debian but there are numerous cons.

Absolutely the same I can say about LizardFS.

> 
> 
>> The only problem is that they didn't change name of
>> binaries (which in my personal opinion is kind of stupid - moosefs =
>> mfsmaster,mfschunkserver,mfsmount etc, LizardFS should be
>> lfsmaster,lfschunkservder,lfsmount etc.).
> 
> As far as I'm aware that was deliberate choice in order to allow seamless 
> upgrades from MooseFS. I think renaming is planned but it is a low priority 
> thing.

Ok. Two years ago maybe it might sense, but now it is only confusing. You can't switch from MooseFS 2.0+ to LizardFS (and vice versa) because forks are not compatible any more, so why not to change binary names?

> 
> 
>> They [LizardFS] actively develop their product, but much slower than
>> original (my personal opinion, but we check their product regularly).
> 
> Well thankfully it is not a race... Since MooseFS do not have public bug 
> tracker or VCS I supose you might be in unique position to compare speed of 
> development. However independent observer will hardly be able to reach the 
> same conclusion.

You are right, but it doesn't matter. The think is that we have totally different goals. In MooseFS we want to have very sturdy, reliable product. This is why I don't want to publish unfinished code and this is why we don't want to have public code repository. I'm considering making public code mirror on GitHub or similar place, but with committed only published sources, not every change I made. Yes, we do not have public bug tracker. Now we use just sourceforge list. Nevertheless this is good idea to have public bug tracker. We will do it ASAP. I can understand necessity of having such. I've just recently found a bug in fuse and didn't know where to post it. The same might be with MooseFS.

Do you know why people choose MooseFS? We have a lot of similar products. We have GlusterFS, Ceph, BeeGFS, XtreemeFS etc. All our clients claim that they have chosen MooseFS because of stability and reliability, not number of features. This is File System we are talking about, not some christmas toy. This is why I only accepted code from community that didn't have issues (at least obvious). LizardFS is full of such small issues. I know many scenarios in LizardFS which lead to data loss/corruption. Sometimes it is not good idea to accept everything to your code. We are very open to our community and we do everything to make our users happy (yes users - users who use GPL version and who don't pay us). For me all GPL users are as much important as PRO users.

This is funny, but especially people from Debian should understand that in computer systems stability is more important than features.

> 
> 
>> This is a fork,
>> so they develop this product in different way and of course it also means
>> that in some rare cases they product is even better than MooseFS. So I
>> absolutely agree that for sure there are some people who will find
>> LizardFS better than MooseFS and therefore both projects should be
>> available in the package repositories.
> 
> I disagree. For Debian there will be undesirable overhead from having two 
> similar software products. It would also cause needless confusion for our 
> users. When choosing between the two I give preference to LizardFS due to 
> greater potential and development transparency. At this time introducing 
> MooseFS to Debian is unnecessary but situation may change in the future.

The think is that we have a lot of users (probably more than LizardFS - of course it's unprovable) and all of them should have right to choose. This is called democracy. MooseFS is available in almost all other OS'es in standard ports/packages.

> 
> 
>> We know everything about LizardFS. Whole company is run by the guy who was
>> fired from our company (he was not a developer, but our it administrator,
>> so he knew MooseFS very well). He "stole" two developers from our company
>> (they never were involved in moosefs developing, but they also knew this
>> product). The whole story is not very nice, but leave it. This is their
>> right to make fork of actively developed project. As you wrote - they have
>> their own investor and therefore their own goal. I do not care about it.
> 
> Thank you for interesting insights about projects history.

There is more. And it's pity that you even help them.

> 
> -- 
> Best wishes,
> Dmitry Smirnov.
> 
> ---
> 
> It is a mistake to try to look too far ahead. The chain of destiny can only
> be grasped one link at a time.
>        -- Winston Churchill

-- 
Regards,
Jakub Kruszona-Zawadzki
- - - - - - - - - - - - - - - -
Segmentation fault (core dumped)
Phone: +48 602 212 039

Message #44 received at 810822@bugs.debian.org (full text, mbox, reply):

From: Dmitry Smirnov <onlyjob@debian.org>

To: Jakub Kruszona-Zawadzki <jakub.kruszona@gemius.com>

Cc: MooseFS Technical Support Department <dwt@moosefs.com>, László Böszörményi (GCS) <gcs@debian.org>, Piotr Robert Konopelko <piotr.konopelko@moosefs.com>, 810822@bugs.debian.org

Subject: Re: Bug#810822: ITP: MooseFS Debian repository contains a fork of MooseFS

Date: Mon, 18 Jan 2016 18:25:19 +1100

[Message part 1 (text/plain, inline)]

On Mon, 18 Jan 2016 06:53:05 AM Jakub Kruszona-Zawadzki wrote:
> On 15 Jan, 2016, at 15:04, Dmitry Smirnov <onlyjob@debian.org> wrote:
> > Except for promoting MooseFS I do not see any benefits from introducing
> > MooseFS to Debian but there are numerous cons.
> 
> Absolutely the same I can say about LizardFS.

Can you be more specific please?


> > As far as I'm aware that was deliberate choice in order to allow seamless
> > upgrades from MooseFS. I think renaming is planned but it is a low
> > priority thing.
> 
> Ok. Two years ago maybe it might sense, but now it is only confusing. You
> can't switch from MooseFS 2.0+ to LizardFS (and vice versa) because forks
> are not compatible any more, so why not to change binary names?

That's right but it is not a big problem since one would hardly use MooseFS 
and LizardFS binaries together. There might be some secondary benefits as 
compatible utility names could be used by user scripts (that would have to be 
adjusted for rename). I agree, perhaps these days it would be nice to finish 
renaming but this is a minor thing... Let's not discuss that here any 
further... There should be corresponding LizardFS bug for this.


> The think is that we have totally
> different goals. In MooseFS we want to have very sturdy, reliable product.

This is unfair to suggest that LizardFS do not strive to have reliable 
product.


> This is why I don't want to publish unfinished code and this is why we
> don't want to have public code repository. I'm considering making public
> code mirror on GitHub or similar place, but with committed only published
> sources, not every change I made.

LizardFS put all contributions - even minor commits through Gerrit code 
review. Their commits are conservative - not too frequent but certainly 
careful.

There is no harm to your users if your VCS is publicly accessible.
Testers and external developers might find it useful and regular users should 
use stable releases.

There might be another reason: MySQL do not expose their VCS because they do 
not want to publish their proprietary code. I suspect MooseFS might have 
similar concerns.


> Yes, we do not have public bug tracker.
> Now we use just sourceforge list. Nevertheless this is good idea to have
> public bug tracker. We will do it ASAP. I can understand necessity of
> having such. I've just recently found a bug in fuse and didn't know where
> to post it. The same might be with MooseFS.

Indeed bug tracker will be very useful even if you only use it internally.
I recommend to consider Redmine which is used by Ceph, Opennebula and many 
others.


> Do you know why people choose MooseFS? We have a lot of similar products.
> We have GlusterFS, Ceph, BeeGFS, XtreemeFS etc. All our clients claim that
> they have chosen MooseFS because of stability and reliability, not number
> of features. This is File System we are talking about, not some christmas
> toy.

While features are also important my own experience confirms that Ceph is 
rubbish and XtreemFS and others are not far away. MooseFS is superiour to all 
those storage systems but its governance made it impossible for me to choose 
it. Hence I'm in favour of LizardFS.


> This is why I only accepted code from community that didn't have
> issues (at least obvious).
> LizardFS is full of such small issues. I know
> many scenarios in LizardFS which lead to data loss/corruption. Sometimes
> it is not good idea to accept everything to your code.

It would be truly great if you could support your claims. You seems to 
suppose that LizardFS bugs were introduced by careless community 
contributions but I do not recall even single case where LizardFS bug was not 
originated either from old MooseFS code or from LizardFS's own development.
Certainly community is not to be blamed for bugs in LizardFS.


> We are very open to
> our community and we do everything to make our users happy (yes users -
> users who use GPL version and who don't pay us). For me all GPL users are
> as much important as PRO users.

That's very nice. Although contributing to MooseFS without VCS and bug 
tracker is not easy...


> The think is that we have a lot of users (probably more than LizardFS - of
> course it's unprovable)

It is entirely possible that MooseFS still have more customers even despite 
loss of those who prefer LizardFS. But popularity is a poor criteria. How is 
it suppose to influence our decision?


> and all of them should have right to choose. This is called democracy.

You seems to mistake freedom of choice for democracy. Democracy is not what 
we use to justify introducing of software to Debian. Also see Mencken's quote 
about democracy below.


> MooseFS is available in almost all other OS'es in standard ports/packages.

MooseFS packaging is immature and do not meet Debian standards of quality.


> > Thank you for interesting insights about projects history.
> 
> There is more. And it's pity that you even help them.

I'm not sorry for helping LizardFS and I do not regret investing my time in 
LizardFS. (For the record I do regret time I wasted helping Ceph). I believe 
LizardFS is worthy of help as they seems to be doing the right thing. They do 
care for their product and I think that's why they've forked MooseFS in first 
place. It remains to be seen if they'll be able to do better job than MooseFS 
team but so far LizardFS team seems to be doing things properly with care for 
best practice etc.

-- 
All the best,
 Dmitry Smirnov.

---

Democracy is a pathetic belief in the collective wisdom of individual
ignorance.
        -- H. L. Mencken

[signature.asc (application/pgp-signature, inline)]

Message #49 received at 810822@bugs.debian.org (full text, mbox, reply):

From: Dmitry Smirnov <onlyjob@debian.org>

To: Jakub Kruszona-Zawadzki <jakub.kruszona@gemius.com>, Piotr Robert Konopelko <piotr.konopelko@moosefs.com>

Cc: 810822@bugs.debian.org, MooseFS Technical Support Department <dwt@moosefs.com>

Subject: Re: Bug#810822: ITP: MooseFS

Date: Mon, 18 Jan 2016 19:10:01 +1100

[Message part 1 (text/plain, inline)]

On Mon, 18 Jan 2016 07:12:04 AM Jakub Kruszona-Zawadzki wrote:
> We may make official clone of GPL'ed version on GitHub if it change
> anything.

I'm sure it would be very helpful in long term.


> 99.9% of my time I spend on GPL'ed version. Now there is
> only one feature in PRO version that is not available in GPL version. We
> do not plan to change that (maybe in the future exchange this feature to
> another).

Fair enough. Thank you for explanation.


> > Debian already have MooseFS' fork -- LizardFS. To my knowledge at the
> > moment MooseFS do not offer noticeable advantages over LizardFS while
> > the latter seems to have slightly more features.
> 
> You are very wrong. We have users who switched from LizardFS. Stability,
> efficiency for example. Quite important advantages. Not for everyone, but
> at least for serious users.

I understand your passion for MooseFS but I am not convinced.


> > For quite a while LizardFS is developed with community using public VCS
> > and bug tracker (GitHub) as well as Gerrit code review system and
> > continuous integration system. LizardFS have more development
> > transparency than MooseFS ever had.
> 
> And in case of file system it is not good idea.
 
Strongly disagreed. Transparency is very important. Code review is important. 
VCS is useful. Continuous integration helps to test every change. Community 
are those who are interested to help improving your product and share 
experience so you won't end up alone. Neglecting all those won't get you 
anywhere...


> > Please note that IMHO MooseFS versus LizardFS situation have many
> > similarities with MySQL vs. MariaDB situation where poor Oracle's
> > governance and focus on proprietary addons discourage community from
> > working with them. (MySQL is not as bad as MooseFS because MySQL have
> > public bug tracker).
> You are not right. MariaDB is developed by original author of MySQL, so the
> spirit of MySQL now is in MariaDB.

I'm talking about pragmatic backporting of bugfixes not abstract "spirit of 
MySQL" or fan club of the original founder.


> I invented MooseFS and I'm still developing MooseFS.

I can't thank you enough for your great work. Thank you sincerely for the 
amazing job you have done.


> Also in terms of
> money. We do not make money on MooseFS. The PRO version is just for us to
> help developing it at all. Let's say that thanks to pro version of MooseFS
> we are able to make GPL version of MooseFS. Thanks to that I'm able to
> work 8 hours a day developing MooseFS, so we made PRO version to actually
> improve developing of GPL version.

Fair enough.


> > As I object to introduction of MooseFS to Debian I would object to
> > introduction of MySQL if MariaDB were already available.
> 
> Why? People should have right to choose.

I've already explained some strong reasons. Diversity and choice are not 
necessary good as there is harm from fragmentation and duplication of 
efforts. Every package have maintenance cost. So far I don't recognise 
benefits from introducing almost similar competitive software merely to 
encourage confusion among our users. I'd like to be convinced that MooseFS is 
superiour but I don't see enough supporting evidence.
In some areas MooseFS is worse, at least due to lack of VCS and bug tracker.


> > With all due respect to MooseFS's former innovations and legacy I think
> > for now it would be best to refrain from debianising MooseFS and
> > re-evaluate situation in the future if there will be any development.
> 
> What kind of "development" are you talking about?

Situational. I meant if/when situation is changed.

-- 
Regards,
 Dmitry Smirnov.

---

Free speech is the bedrock of liberty and a free society. And yes, it
includes the right to blaspheme and offend.
        -- Ayaan Hirsi Ali, 2010

[signature.asc (application/pgp-signature, inline)]

Message #54 received at 810822@bugs.debian.org (full text, mbox, reply):

From: Jakub Kruszona-Zawadzki <jakub.kruszona@gemius.com>

To: Dmitry Smirnov <onlyjob@debian.org>

Cc: MooseFS Technical Support Department <dwt@moosefs.com>, "László Böszörményi (GCS)" <gcs@debian.org>, Piotr Robert Konopelko <piotr.konopelko@moosefs.com>, 810822@bugs.debian.org

Subject: Re: Bug#810822: ITP: MooseFS Debian repository contains a fork of MooseFS

Date: Mon, 18 Jan 2016 09:16:25 +0100

On 18 Jan, 2016, at 8:25, Dmitry Smirnov <onlyjob@debian.org> wrote:

> On Mon, 18 Jan 2016 06:53:05 AM Jakub Kruszona-Zawadzki wrote:
>> On 15 Jan, 2016, at 15:04, Dmitry Smirnov <onlyjob@debian.org> wrote:
>>> Except for promoting MooseFS I do not see any benefits from introducing
>>> MooseFS to Debian but there are numerous cons.
>> 
>> Absolutely the same I can say about LizardFS.
> 
> Can you be more specific please?
> 

To have "only" LizardFS in your repository you promote LizardFS and I don't know why?

> 
>>> As far as I'm aware that was deliberate choice in order to allow seamless
>>> upgrades from MooseFS. I think renaming is planned but it is a low
>>> priority thing.
>> 
>> Ok. Two years ago maybe it might sense, but now it is only confusing. You
>> can't switch from MooseFS 2.0+ to LizardFS (and vice versa) because forks
>> are not compatible any more, so why not to change binary names?
> 
> That's right but it is not a big problem since one would hardly use MooseFS 
> and LizardFS binaries together. There might be some secondary benefits as 
> compatible utility names could be used by user scripts (that would have to be 
> adjusted for rename). I agree, perhaps these days it would be nice to finish 
> renaming but this is a minor thing... Let's not discuss that here any 
> further... There should be corresponding LizardFS bug for this.

Ok.

> 
> 
>> The think is that we have totally
>> different goals. In MooseFS we want to have very sturdy, reliable product.
> 
> This is unfair to suggest that LizardFS do not strive to have reliable 
> product.
> 

Maybe they want, but they do not have it. Number of issues in they github is huge (of course since we do not have public bug track we can't compare :) )

But we will change that. There will be official source tree of GPL'ed MooseFS in GitHub.

> 
>> This is why I don't want to publish unfinished code and this is why we
>> don't want to have public code repository. I'm considering making public
>> code mirror on GitHub or similar place, but with committed only published
>> sources, not every change I made.
> 
> LizardFS put all contributions - even minor commits through Gerrit code 
> review. Their commits are conservative - not too frequent but certainly 
> careful.

No comment.

> 
> There is no harm to your users if your VCS is publicly accessible.
> Testers and external developers might find it useful and regular users should 
> use stable releases.

I agree. As I mentioned we will change that. There will be official MooseFS source code on GitHub.

> 
> There might be another reason: MySQL do not expose their VCS because they do 
> not want to publish their proprietary code. I suspect MooseFS might have 
> similar concerns.

We will not publish our proprietary code (as for now), but it doesn't mean that we will not publish GPL'ed MooseFS. As I wrote. GPL'ed version is almost full MooseFS. I even plan to publish master failover in GPL'ed version (not full HA as we have it in PRO, but failover as in LizardFS). I have to convince our investor, but I hope that it will be doable. I'm very pro free software and I think that there should be no "pro" version of MooseFS (my personal opinion), but life is sometimes brutal and unfair.

> 
> 
>> Yes, we do not have public bug tracker.
>> Now we use just sourceforge list. Nevertheless this is good idea to have
>> public bug tracker. We will do it ASAP. I can understand necessity of
>> having such. I've just recently found a bug in fuse and didn't know where
>> to post it. The same might be with MooseFS.
> 
> Indeed bug tracker will be very useful even if you only use it internally.
> I recommend to consider Redmine which is used by Ceph, Opennebula and many 
> others.

Yes. We use Redmine internally, but I agree that having public bug tracking is the best, because all users will have access to that.

> 
> 
>> Do you know why people choose MooseFS? We have a lot of similar products.
>> We have GlusterFS, Ceph, BeeGFS, XtreemeFS etc. All our clients claim that
>> they have chosen MooseFS because of stability and reliability, not number
>> of features. This is File System we are talking about, not some christmas
>> toy.
> 
> While features are also important my own experience confirms that Ceph is 
> rubbish and XtreemFS and others are not far away.

This time I agree with you in 100% :)

> MooseFS is superiour to all 
> those storage systems but its governance made it impossible for me to choose 
> it. Hence I'm in favour of LizardFS.

I hope that it will change some day. Did you even try to test MooseFS 3.0? Compare to LizardFS? 

If you need PRO licence to do some test then let me know - We will send it.

> 
> 
>> This is why I only accepted code from community that didn't have
>> issues (at least obvious).
>> LizardFS is full of such small issues. I know
>> many scenarios in LizardFS which lead to data loss/corruption. Sometimes
>> it is not good idea to accept everything to your code.
> 
> It would be truly great if you could support your claims. You seems to 
> suppose that LizardFS bugs were introduced by careless community 
> contributions but I do not recall even single case where LizardFS bug was not 
> originated either from old MooseFS code or from LizardFS's own development.
> Certainly community is not to be blamed for bugs in LizardFS.

Some bugs in LizardFS originated from old MooseFS (I know because I fixed them), some are their own. It's normal. MooseFS also have bugs (I hope only small ones - but you never know :) ).

Simple example of LizardFS wrongdoings. First change they made was to accept chunks with bigger version than version set in master. Chunk version mismatch was in this case for a reason. This can lead to acceptance chunk with wrong data inside. They change it because there were problems with version synchronization (problem originated from original MooseFS). To fix that you should fix synchronization, not just accept everything.

Believe me this is only one simple example. I know what I fixed in MooseFS in those years since 1.6. They didn't fix almost any of these.

> 
> 
>> We are very open to
>> our community and we do everything to make our users happy (yes users -
>> users who use GPL version and who don't pay us). For me all GPL users are
>> as much important as PRO users.
> 
> That's very nice. Although contributing to MooseFS without VCS and bug 
> tracker is not easy...

You are right. We will change that.

> 
> 
>> The think is that we have a lot of users (probably more than LizardFS - of
>> course it's unprovable)
> 
> It is entirely possible that MooseFS still have more customers even despite 
> loss of those who prefer LizardFS. But popularity is a poor criteria. How is 
> it suppose to influence our decision?
> 

Maybe because huge part of our users are also Debian users and you may want to make their life easier.

> 
>> and all of them should have right to choose. This is called democracy.
> 
> You seems to mistake freedom of choice for democracy. Democracy is not what 
> we use to justify introducing of software to Debian. Also see Mencken's quote 
> about democracy below.

:)
ok here I totally agree. I don't know if you are aware of this or not, but we have in Poland new government chosen in democratic poll, and they are just ... stupid (horribly stupid and radical).

So call it freedom not democracy. Your (Debian) users should have freedom to choose.

> 
> 
>> MooseFS is available in almost all other OS'es in standard ports/packages.
> 
> MooseFS packaging is immature and do not meet Debian standards of quality.

I didn't know that. Can you help us to improve that (I understand that you do not have time and don't want to involve, but maybe there is some documentation, some hints - how to improve that)?

> 
> 
>>> Thank you for interesting insights about projects history.
>> 
>> There is more. And it's pity that you even help them.
> 
> I'm not sorry for helping LizardFS and I do not regret investing my time in 
> LizardFS. (For the record I do regret time I wasted helping Ceph). I believe 
> LizardFS is worthy of help as they seems to be doing the right thing.

This is your decision who you are helping. If you believe in them then help them, but at least try not to be against us. I'm still inventor of MooseFS and I hope that I still make a good job with MooseFS.

> They do 
> care for their product and I think that's why they've forked MooseFS in first 
> place.

Of course they care, but it doesn't mean that they make success in that.
As I know (and I know a lot about this, but not all of course) there were other reasons behind fork, but it is another story.

> It remains to be seen if they'll be able to do better job than MooseFS 
> team but so far LizardFS team seems to be doing things properly with care for 
> best practice etc.
> 
> -- 
> All the best,
> Dmitry Smirnov.
> 
> ---
> 
> Democracy is a pathetic belief in the collective wisdom of individual
> ignorance.
>        -- H. L. Mencken

-- 
Pozdrawiam,
Jakub Kruszona-Zawadzki
- - - - - - - - - - - - - - - -
Segmentation fault (core dumped)
Tel. +48 602 212 039

Message #59 received at 810822@bugs.debian.org (full text, mbox, reply):

From: Dmitry Smirnov <onlyjob@debian.org>

To: Jakub Kruszona-Zawadzki <jakub.kruszona@gemius.com>

Cc: MooseFS Technical Support Department <dwt@moosefs.com>, László Böszörményi (GCS) <gcs@debian.org>, Piotr Robert Konopelko <piotr.konopelko@moosefs.com>, 810822@bugs.debian.org

Subject: Re: Bug#810822: ITP: MooseFS Debian repository contains a fork of MooseFS

Date: Mon, 18 Jan 2016 20:04:09 +1100

[Message part 1 (text/plain, inline)]

On Mon, 18 Jan 2016 09:16:25 AM Jakub Kruszona-Zawadzki wrote:
> > This is unfair to suggest that LizardFS do not strive to have reliable
> > product.
> 
> Maybe they want, but they do not have it. Number of issues in they github
> is huge (of course since we do not have public bug track we can't compare
> :) )

Number of bugs is not a metric of quality. Remember that bugs are also items 
on developers' TO DO list.
I reported more LizardFS bugs than anyone else (up to 100) and I'm using 
LizardFS because it survived vigorous testing. Bugs are also different: some 
are feature requests, some are not critical for data integrity etc.
I'm not aware of release-critical bugs without fix but of course there might 
be some.


> Hence I'm in favour of LizardFS.
> 
> I hope that it will change some day. Did you even try to test MooseFS 3.0?
> Compare to LizardFS?

So far I've seen no compelling reasons to switch away from LizardFS - not 
until MooseFS become more open. At the moment IMHO LizardFS is the only 
choice (and GfarmFS is close second). Also I have no time to try MooseFS. 
Maybe one day I'll try it but I'll have to justify the effort...


> If you need PRO licence to do some test then let me know - We will send it.

Thank you for your kind offer but no. :) I will only evaluate what's free and 
suitable for Debian.


> Simple example of LizardFS wrongdoings. First change they made was to
> accept chunks with bigger version than version set in master. Chunk
> version mismatch was in this case for a reason. This can lead to
> acceptance chunk with wrong data inside. They change it because there were
> problems with version synchronization (problem originated from original
> MooseFS). To fix that you should fix synchronization, not just accept
> everything.

Interesting, thank you.


> Maybe because huge part of our users are also Debian users and you may want
> to make their life easier.

Makes sense.


> I don't know if you are aware of this or not, but
> we have in Poland new government chosen in democratic poll, and they are
> just ... stupid (horribly stupid and radical).

I feel your pain. I feel ashamed of our Australian government. :(

Popularity polls are poor foundation for decision making. But as Winston 
Churchill once said, "It has been said that democracy is the worst form of 
government except all the others that have been tried."...


> I didn't know that. Can you help us to improve that (I understand that you
> do not have time and don't want to involve, but maybe there is some
> documentation, some hints - how to improve that)?

Here you can find some generic links to documents describing packaging for 
Debian:

    http://mentors.debian.net/qa

Lintian checks (including pedantic and experimental ones) provide useful 
hints for areas that can benefit from improvements.

Also you can refer to official LizardFS packaging which have numerous 
improvements over "upstream" packaging: 

    https://anonscm.debian.org/cgit/collab-maint/lizardfs.git

Feel free to ask questions in mentor's mail list. If you CC to me I'll try to 
answer as well if time allows.


> This is your decision who you are helping. If you believe in them then help
> them, but at least try not to be against us. I'm still inventor of MooseFS
> and I hope that I still make a good job with MooseFS.

I'm not agains you or MooseFS. Even though at the moment I'm in favour of 
LizardFS I'm sure LizardFS would not be where it is now without solid 
foundation of MooseFS.

-- 
All the best,
 Dmitry Smirnov.

---

Odious ideas are not entitled to hide from criticism behind the human
shield of their believers' feelings.
        -- Richard Stallman

[signature.asc (application/pgp-signature, inline)]

Message #64 received at 810822@bugs.debian.org (full text, mbox, reply):

From: Jakub Kruszona-Zawadzki <jakub.kruszona@gemius.com>

To: Dmitry Smirnov <onlyjob@debian.org>

Cc: MooseFS Technical Support Department <dwt@moosefs.com>, "László Böszörményi (GCS)" <gcs@debian.org>, Piotr Robert Konopelko <piotr.konopelko@moosefs.com>, 810822@bugs.debian.org

Subject: Re: Bug#810822: ITP: MooseFS Debian repository contains a fork of MooseFS

Date: Mon, 18 Jan 2016 10:51:54 +0100

On 18 Jan, 2016, at 10:04, Dmitry Smirnov <onlyjob@debian.org> wrote:

> On Mon, 18 Jan 2016 09:16:25 AM Jakub Kruszona-Zawadzki wrote:
>>> This is unfair to suggest that LizardFS do not strive to have reliable
>>> product.
>> 
>> Maybe they want, but they do not have it. Number of issues in they github
>> is huge (of course since we do not have public bug track we can't compare
>> :) )
> 
> Number of bugs is not a metric of quality.

Really?

> Remember that bugs are also items 
> on developers' TO DO list.
> I reported more LizardFS bugs than anyone else (up to 100) and I'm using 
> LizardFS because it survived vigorous testing.

Did you ever tried huge installations? 3PB, 100mln of files?

> Bugs are also different: some 
> are feature requests, some are not critical for data integrity etc.
> I'm not aware of release-critical bugs without fix but of course there might 
> be some.
> 
> 
>> Hence I'm in favour of LizardFS.

Your choice. I don't have problem with that, but please don't be against us.

>> 
>> I hope that it will change some day. Did you even try to test MooseFS 3.0?
>> Compare to LizardFS?
> 
> So far I've seen no compelling reasons to switch away from LizardFS - not 
> until MooseFS become more open. At the moment IMHO LizardFS is the only 
> choice (and GfarmFS is close second). Also I have no time to try MooseFS. 
> Maybe one day I'll try it but I'll have to justify the effort...

So you shouldn't say bad words about MooseFS. We test LizardFS (and others - like Ceph, Gluster etc.) on regular basis.

I understand that it is time-consuming, but I hope that you will find time in the future and give it a try.

> 
> 
>> If you need PRO licence to do some test then let me know - We will send it.
> 
> Thank you for your kind offer but no. :) I will only evaluate what's free and 
> suitable for Debian.

ok. :)

> 
> 
>> Simple example of LizardFS wrongdoings. First change they made was to
>> accept chunks with bigger version than version set in master. Chunk
>> version mismatch was in this case for a reason. This can lead to
>> acceptance chunk with wrong data inside. They change it because there were
>> problems with version synchronization (problem originated from original
>> MooseFS). To fix that you should fix synchronization, not just accept
>> everything.
> 
> Interesting, thank you.
> 
> 
>> Maybe because huge part of our users are also Debian users and you may want
>> to make their life easier.
> 
> Makes sense.
> 
> 
>> I don't know if you are aware of this or not, but
>> we have in Poland new government chosen in democratic poll, and they are
>> just ... stupid (horribly stupid and radical).
> 
> I feel your pain. I feel ashamed of our Australian government. :(
> 
> Popularity polls are poor foundation for decision making. But as Winston 
> Churchill once said, "It has been said that democracy is the worst form of 
> government except all the others that have been tried."...

Exactly.

> 
> 
>> I didn't know that. Can you help us to improve that (I understand that you
>> do not have time and don't want to involve, but maybe there is some
>> documentation, some hints - how to improve that)?
> 
> Here you can find some generic links to documents describing packaging for 
> Debian:
> 
>    http://mentors.debian.net/qa
> 
> Lintian checks (including pedantic and experimental ones) provide useful 
> hints for areas that can benefit from improvements.
> 
> Also you can refer to official LizardFS packaging which have numerous 
> improvements over "upstream" packaging: 
> 
>    https://anonscm.debian.org/cgit/collab-maint/lizardfs.git
> 
> Feel free to ask questions in mentor's mail list. If you CC to me I'll try to 
> answer as well if time allows.

Thanks a lot. This is very helpful, we will check that. I remember porting to official freebsd ports - PITA, but what to do. There are rules and we (developers) should obey them.

> 
> 
>> This is your decision who you are helping. If you believe in them then help
>> them, but at least try not to be against us. I'm still inventor of MooseFS
>> and I hope that I still make a good job with MooseFS.
> 
> I'm not agains you or MooseFS. Even though at the moment I'm in favour of 
> LizardFS I'm sure LizardFS would not be where it is now without solid 
> foundation of MooseFS.

Thanks for saying that, because at the beginning I had a feeling that you are against us.

We are still making this solid foundation (i hope :) ).

> 
> -- 
> All the best,
> Dmitry Smirnov.
> 
> ---
> 
> Odious ideas are not entitled to hide from criticism behind the human
> shield of their believers' feelings.
>        -- Richard Stallman

-- 
Regards,
Jakub Kruszona-Zawadzki
- - - - - - - - - - - - - - - -
Segmentation fault (core dumped)
Phone: +48 602 212 039

Send a report that this bug log contains spam.