https://www.coria-cfd.fr/index.php?title=Quotas_on_irene&feed=atom&action=historyQuotas on irene - Revision history2024-03-29T09:25:35ZRevision history for this page on the wikiMediaWiki 1.26.2https://www.coria-cfd.fr/index.php?title=Quotas_on_irene&diff=3811&oldid=prevLartigue: Created page with "== What's the problem? == Sometimes, on Irene Joliot-Curie (the last TGCC supercomputer), you will be confronted to the following error: '''''error writing ‘XXXX’: Disk..."2018-12-05T09:37:52Z<p>Created page with "== What's the problem? == Sometimes, on Irene Joliot-Curie (the last TGCC supercomputer), you will be confronted to the following error: '''''error writing ‘XXXX’: Disk..."</p>
<p><b>New page</b></p><div>== What's the problem? ==<br />
<br />
Sometimes, on Irene Joliot-Curie (the last TGCC supercomputer), you will be confronted to the following error:<br />
<br />
'''''error writing ‘XXXX’: Disk quota exceeded'''''<br />
<br />
This problem is actually more complex than it seems.<br />
This goal of this page is to provide the explanation of this issue and how to solve it.<br />
<br />
<br />
== Groups ==<br />
<br />
On Linux, a user is defined by a number, which is set in /etc/passwd. For exemple, on my machine, I am the user 1001:<br />
<br />
''> cat /etc/passwd | grep lartigue''<br />
<br />
'''lartigue:x:1001:1001::/home/lartigue:/bin/bash'''<br />
<br />
As you can see, the number 1001 is printed two times on this line: the first one corresponds to my 'User ID', the second one corresponds to my 'Group ID': these numbers are not necessarily identical.<br />
This means that each user is associated to a group, known as its 'principal group'.<br />
<br />
However, a user can be associated to many other groups (which are known as ‘secondary groups’).<br />
For exemple, on irene, my groups are:<br />
<br />
''> groups''<br />
<br />
'''coria avbp ra0381 gen6880 gch0011 ra1113 intelmpi gen7345 gen10161 ra3999 gch0305 cont005'''<br />
<br />
The first group that appears on this line is my principal group, the others are my secondary groups.<br />
<br />
<br />
== Creating files/directories ==<br />
<br />
On Linux, each file on the filesystem possesses some metadata: its size, its creation date, to which user and to which group it belongs and what are the associated permissions.<br />
When a user creates a file (via cp/scp/touch/rsync/vi/...) this file is automatically associated to this user and to its principal group.<br />
<br />
For example when I create a file on irene:<br />
<br />
''> touch /tmp/toto''<br />
<br />
'' > ls -l /tmp/toto''<br />
<br />
'''-rw-r-----. 1 lartigug coria 0 Nov 5 08:38 /tmp/toto'''<br />
<br />
This file is empty (size=0), it has been created on Nov. 5th at 8:38AM and is associated to me (user=lartigug) and to my principal group (group=coria). <br />
<br />
Moreover, as my umask is set to 0027, its permissions are set to '''-rw-r-----'''<br />
<br />
<br />
== The real Issue ==<br />
<br />
It is actually quite simple to understand: on every production machine, there are some quota associated to users and groups.<br />
This means that the amount of data that belongs to you or to one of your group can not go beyond a certain limit.<br />
<br />
Now, if you transfer a lot of files (scp/rsync) or if you create a lot of file (during a simulation), all these files will belong to your principal group. And if there is a quota on this group, you will get the nasty error '''''error writing ‘XXXX’: Disk quota exceeded'''''.<br />
<br />
<br />
== How to solve this issue ==<br />
<br />
Well, the idea is quite simple: you must create the files with a different group for which the quota is not exceeded…<br />
<br />
For example on irene, the quota on group gen6880 is much larger than on the group coria.<br />
<br />
But HOW?<br />
<br />
<br />
== Technical solution #1 ==<br />
<br />
If the files are already there but you want to change their group, just use the classical ''chgrp [-R]'' command (with -R to go recursive).<br />
<br />
''> mkdir /tmp/dir_test''<br />
<br />
''> ls -la /tmp/dir_test''<br />
<br />
'''drwxr-x---. 2 lartigug coria 6 Nov 5 08:58 .'''<br />
<br />
''chgrp gen6880 /tmp/dir_test''<br />
<br />
'''drwxr-x---. 2 lartigug gen6880 6 Nov 5 08:58 .'''<br />
<br />
<br />
== Technical solution #2 ==<br />
<br />
You can change the default behaviour of directories so that new files are created with different ownership.<br />
<br />
If I create a file in the previous directory /tmp/dir_test (group=gen6880), this file is created with the classical ownership lartigug:coria<br />
<br />
''> touch /tmp/dir_test/file1''<br />
<br />
''> ls -l /tmp/dir_test/file1''<br />
<br />
'''-rw-r-----. 1 lartigug coria 0 Nov 5 09:04 /tmp/dir_test/file1'''<br />
<br />
<br />
Now let me ''setgid'' the directory /tmp/dir_test. This is achieved with the following command:<br />
<br />
''> chmod g+s /tmp/dir_test''<br />
<br />
Now check what happend:<br />
<br />
''> ls -la /tmp/dir_test''<br />
<br />
'''drwxr-s---. 2 lartigug gen6880 32 Nov 5 09:05 .'''<br />
<br />
The ''s'' indicates that the directory is now setgid.<br />
This means that each new file/directory will be created with ownership lartigug:gen6880<br />
<br />
Let’s check this:<br />
<br />
''> touch /tmp/dir_test/file2''<br />
<br />
''> ls -l /tmp/dir_test/file2''<br />
<br />
'''-rw-r-----. 1 lartigug gen6880 0 Nov 5 09:05 file2'''<br />
<br />
Moreover when creating a new sub-directory, it will keep its setgid property!<br />
<br />
''> mkdir subdir''<br />
<br />
''> ls -l''<br />
<br />
'''drwxr-s---. 2 lartigug gen6880 6 Nov 5 09:15 subdir'''<br />
<br />
<br />
<br />
== Technical solution #3 ==<br />
<br />
Some difficulties may arise when you transfer files from your computer to irene.<br />
<br />
<br />
=== Use tar & scp ===<br />
<br />
first tar you directories<br />
transfer the tar<br />
untar<br />
<br />
This will keep the right ownership and setgid property in all your directories/subdirectories.<br />
<br />
<br />
=== Use rsync with options ===<br />
<br />
Use rsync but add the following options to modify its default behaviour:<br />
<br />
rsync --chmod=g+s --chown=:gen6880 …….<br />
<br />
<br />
=== Use scp -r ===<br />
<br />
If you do so, you '''MUST''' then do a ''chmod -R g+s'' on your directory because the setgid will not propagate in the subdirectories.<br />
<br />
<br />
== References ==<br />
<br />
http://www.linuxaddict.fr/index.php/2018/06/04/les-droits-speciaux-sous-gnu-linux-setuid-setgid-sticky-bit-et-umask/<br />
<br />
https://tech.feub.net/2008/03/setuid-setgid-et-sticky-bit/#</div>Lartigue