Lartigue: Created page with "== What's the problem? == Sometimes, on Irene Joliot-Curie (the last TGCC supercomputer), you will be confronted to the following error: '''''error writing ‘XXXX’: Disk..."

2018-12-05T09:37:52Z

Created page with "== What's the problem? == Sometimes, on Irene Joliot-Curie (the last TGCC supercomputer), you will be confronted to the following error: '''''error writing ‘XXXX’: Disk..."

New page

== What's the problem? ==

Sometimes, on Irene Joliot-Curie (the last TGCC supercomputer), you will be confronted to the following error:

'''''error writing ‘XXXX’: Disk quota exceeded'''''

This problem is actually more complex than it seems.
This goal of this page is to provide the explanation of this issue and how to solve it.

== Groups ==

On Linux, a user is defined by a number, which is set in /etc/passwd. For exemple, on my machine, I am the user 1001:

''> cat /etc/passwd | grep lartigue''

'''lartigue:x:1001:1001::/home/lartigue:/bin/bash'''

As you can see, the number 1001 is printed two times on this line: the first one corresponds to my 'User ID', the second one corresponds to my 'Group ID': these numbers are not necessarily identical.
This means that each user is associated to a group, known as its 'principal group'.

However, a user can be associated to many other groups (which are known as ‘secondary groups’).
For exemple, on irene, my groups are:

''> groups''

'''coria avbp ra0381 gen6880 gch0011 ra1113 intelmpi gen7345 gen10161 ra3999 gch0305 cont005'''

The first group that appears on this line is my principal group, the others are my secondary groups.

== Creating files/directories ==

On Linux, each file on the filesystem possesses some metadata: its size, its creation date, to which user and to which group it belongs and what are the associated permissions.
When a user creates a file (via cp/scp/touch/rsync/vi/...) this file is automatically associated to this user and to its principal group.

For example when I create a file on irene:

''> touch /tmp/toto''

'' > ls -l /tmp/toto''

'''-rw-r-----. 1 lartigug coria 0 Nov 5 08:38 /tmp/toto'''

This file is empty (size=0), it has been created on Nov. 5th at 8:38AM and is associated to me (user=lartigug) and to my principal group (group=coria).

Moreover, as my umask is set to 0027, its permissions are set to '''-rw-r-----'''

== The real Issue ==

It is actually quite simple to understand: on every production machine, there are some quota associated to users and groups.
This means that the amount of data that belongs to you or to one of your group can not go beyond a certain limit.

Now, if you transfer a lot of files (scp/rsync) or if you create a lot of file (during a simulation), all these files will belong to your principal group. And if there is a quota on this group, you will get the nasty error '''''error writing ‘XXXX’: Disk quota exceeded'''''.

== How to solve this issue ==

Well, the idea is quite simple: you must create the files with a different group for which the quota is not exceeded…

For example on irene, the quota on group gen6880 is much larger than on the group coria.

But HOW?

== Technical solution #1 ==

If the files are already there but you want to change their group, just use the classical ''chgrp [-R]'' command (with -R to go recursive).

''> mkdir /tmp/dir_test''

''> ls -la /tmp/dir_test''

'''drwxr-x---. 2 lartigug coria 6 Nov 5 08:58 .'''

''chgrp gen6880 /tmp/dir_test''

'''drwxr-x---. 2 lartigug gen6880 6 Nov 5 08:58 .'''

== Technical solution #2 ==

You can change the default behaviour of directories so that new files are created with different ownership.

If I create a file in the previous directory /tmp/dir_test (group=gen6880), this file is created with the classical ownership lartigug:coria

''> touch /tmp/dir_test/file1''

''> ls -l /tmp/dir_test/file1''

'''-rw-r-----. 1 lartigug coria 0 Nov 5 09:04 /tmp/dir_test/file1'''

Now let me ''setgid'' the directory /tmp/dir_test. This is achieved with the following command:

''> chmod g+s /tmp/dir_test''

Now check what happend:

''> ls -la /tmp/dir_test''

'''drwxr-s---. 2 lartigug gen6880 32 Nov 5 09:05 .'''

The ''s'' indicates that the directory is now setgid.
This means that each new file/directory will be created with ownership lartigug:gen6880

Let’s check this:

''> touch /tmp/dir_test/file2''

''> ls -l /tmp/dir_test/file2''

'''-rw-r-----. 1 lartigug gen6880 0 Nov 5 09:05 file2'''

Moreover when creating a new sub-directory, it will keep its setgid property!

''> mkdir subdir''

''> ls -l''

'''drwxr-s---. 2 lartigug gen6880 6 Nov 5 09:15 subdir'''

== Technical solution #3 ==

Some difficulties may arise when you transfer files from your computer to irene.

=== Use tar & scp ===

first tar you directories
transfer the tar
untar

This will keep the right ownership and setgid property in all your directories/subdirectories.

=== Use rsync with options ===

Use rsync but add the following options to modify its default behaviour:

rsync --chmod=g+s --chown=:gen6880 …….

=== Use scp -r ===

If you do so, you '''MUST''' then do a ''chmod -R g+s'' on your directory because the setgid will not propagate in the subdirectories.

== References ==

http://www.linuxaddict.fr/index.php/2018/06/04/les-droits-speciaux-sous-gnu-linux-setuid-setgid-sticky-bit-et-umask/

https://tech.feub.net/2008/03/setuid-setgid-et-sticky-bit/#

Quotas on irene - Revision history

Lartigue: Created page with "== What's the problem? == Sometimes, on Irene Joliot-Curie (the last TGCC supercomputer), you will be confronted to the following error: '''''error writing ‘XXXX’: Disk..."