Revision 0d0229b5 doc/design-2.1.rst
b/doc/design-2.1.rst | ||
---|---|---|
15 | 15 |
========= |
16 | 16 |
|
17 | 17 |
Ganeti 2.1 will add features to help further automatization of cluster |
18 |
operations, further improbe scalability to even bigger clusters, and
|
|
18 |
operations, further improve scalability to even bigger clusters, and
|
|
19 | 19 |
make it easier to debug the Ganeti core. |
20 | 20 |
|
21 | 21 |
Background |
... | ... | |
725 | 725 |
all instances on the node. |
726 | 726 |
|
727 | 727 |
|
728 |
User-id pool |
|
729 |
~~~~~~~~~~~~ |
|
730 |
|
|
731 |
In order to allow running different processes under unique user-ids |
|
732 |
on a node, we introduce the user-id pool concept. |
|
733 |
|
|
734 |
The user-id pool is a cluster-wide configuration parameter. |
|
735 |
It is a list of user-ids and/or user-id ranges that are reserved |
|
736 |
for running Ganeti processes (including KVM instances). |
|
737 |
The code guarantees that on a given node a given user-id is only |
|
738 |
handed out if there is no other process running with that user-id. |
|
739 |
|
|
740 |
Please note, that this can only be guaranteed if all processes in |
|
741 |
the system - that run under a user-id belonging to the pool - are |
|
742 |
started by reserving a user-id first. That can be accomplished |
|
743 |
either by using the RequestUnusedUid() function to get an unused |
|
744 |
user-id or by implementing the same locking mechanism. |
|
745 |
|
|
746 |
Implementation |
|
747 |
++++++++++++++ |
|
748 |
|
|
749 |
The functions that are specific to the user-id pool feature are located |
|
750 |
in a separate module: ``lib/uidpool.py``. |
|
751 |
|
|
752 |
Storage |
|
753 |
^^^^^^^ |
|
754 |
|
|
755 |
The user-id pool is a single cluster parameter. It is stored in the |
|
756 |
*Cluster* object under the ``uid_pool`` name as a list of integer |
|
757 |
tuples. These tuples represent the boundaries of user-id ranges. |
|
758 |
For single user-ids, the boundaries are equal. |
|
759 |
|
|
760 |
The internal user-id pool representation is converted into a |
|
761 |
string: a newline separated list of user-ids or user-id ranges. |
|
762 |
This string representation is distributed to all the nodes via the |
|
763 |
*ssconf* mechanism. This means that the user-id pool can be |
|
764 |
accessed in a read-only way on any node without consulting the master |
|
765 |
node or master candidate nodes. |
|
766 |
|
|
767 |
Initial value |
|
768 |
^^^^^^^^^^^^^ |
|
769 |
|
|
770 |
The value of the user-id pool cluster parameter can be initialized |
|
771 |
at cluster initialization time using the |
|
772 |
|
|
773 |
``gnt-cluster init --uid-pool <uid-pool definition> ...`` |
|
774 |
|
|
775 |
command. |
|
776 |
|
|
777 |
As there is no sensible default value for the user-id pool parameter, |
|
778 |
it is initialized to an empty list if no ``--uid-pool`` option is |
|
779 |
supplied at cluster init time. |
|
780 |
|
|
781 |
If the user-id pool is empty, the user-id pool feature is considered |
|
782 |
to be disabled. |
|
783 |
|
|
784 |
Manipulation |
|
785 |
^^^^^^^^^^^^ |
|
786 |
|
|
787 |
The user-id pool cluster parameter can be modified from the |
|
788 |
command-line with the following commands: |
|
789 |
|
|
790 |
- ``gnt-cluster modify --uid-pool <uid-pool definition>`` |
|
791 |
- ``gnt-cluster modify --add-uids <uid-pool definition>`` |
|
792 |
- ``gnt-cluster modify --remove-uids <uid-pool definition>`` |
|
793 |
|
|
794 |
The ``--uid-pool`` option overwrites the current setting with the |
|
795 |
supplied ``<uid-pool definition>``, while |
|
796 |
``--add-uids``/``--remove-uids`` adds/removes the listed uids |
|
797 |
or uid-ranges from the pool. |
|
798 |
|
|
799 |
The ``<uid-pool definition>`` should be a comma-separated list of |
|
800 |
user-ids or user-id ranges. A range should be defined by a lower and |
|
801 |
a higher boundary. The boundaries should be separated with a dash. |
|
802 |
The boundaries are inclusive. |
|
803 |
|
|
804 |
The ``<uid-pool definition>`` is parsed into the internal |
|
805 |
representation, sanity-checked and stored in the ``uid_pool`` |
|
806 |
attribute of the *Cluster* object. |
|
807 |
|
|
808 |
It is also immediately converted into a string (formatted in the |
|
809 |
input format) and distributed to all nodes via the *ssconf* mechanism. |
|
810 |
|
|
811 |
Inspection |
|
812 |
^^^^^^^^^^ |
|
813 |
|
|
814 |
The current value of the user-id pool cluster parameter is printed |
|
815 |
by the ``gnt-cluster info`` command. |
|
816 |
|
|
817 |
The output format is accepted by the ``gnt-cluster modify --uid-pool`` |
|
818 |
command. |
|
819 |
|
|
820 |
Locking |
|
821 |
^^^^^^^ |
|
822 |
|
|
823 |
The ``uidpool.py`` module provides a function (``RequestUnusedUid``) |
|
824 |
for requesting an unused user-id from the pool. |
|
825 |
|
|
826 |
This will try to find a random user-id that is not currently in use. |
|
827 |
The algorithm is the following: |
|
828 |
|
|
829 |
1) Randomize the list of user-ids in the user-id pool |
|
830 |
2) Iterate over this randomized UID list |
|
831 |
3) Create a lock file (it doesn't matter if it already exists) |
|
832 |
4) Acquire an exclusive POSIX lock on the file, to provide mutual |
|
833 |
exclusion for the following non-atomic operations |
|
834 |
5) Check if there is a process in the system with the given UID |
|
835 |
6) If there isn't, return the UID, otherwise unlock the file and |
|
836 |
continue the iteration over the user-ids |
|
837 |
|
|
838 |
The user can than start a new process with this user-id. |
|
839 |
Once a process is successfully started, the exclusive POSIX lock can |
|
840 |
be released, but the lock file will remain in the filesystem. |
|
841 |
The presence of such a lock file means that the given user-id is most |
|
842 |
probably in use. The lack of a uid lock file does not guarantee that |
|
843 |
there are no processes with that user-id. |
|
844 |
|
|
845 |
After acquiring the exclusive POSIX lock, ``RequestUnusedUid`` |
|
846 |
always performs a check to see if there is a process running with the |
|
847 |
given uid. |
|
848 |
|
|
849 |
A user-id can be returned to the pool, by calling the |
|
850 |
``ReleaseUid`` function. This will remove the corresponding lock file. |
|
851 |
Note, that it doesn't check if there is any process still running |
|
852 |
with that user-id. The removal of the lock file only means that there |
|
853 |
are most probably no processes with the given user-id. This helps |
|
854 |
in speeding up the process of finding a user-id that is guaranteed to |
|
855 |
be unused. |
|
856 |
|
|
857 |
There is a convenience function, called ``ExecWithUnusedUid`` that |
|
858 |
wraps the execution of a function (or any callable) that requires a |
|
859 |
unique user-id. ``ExecWithUnusedUid`` takes care of requesting an |
|
860 |
unused user-id and unlocking the lock file. It also automatically |
|
861 |
returns the user-id to the pool if the callable raises an exception. |
|
862 |
|
|
863 |
Code examples |
|
864 |
+++++++++++++ |
|
865 |
|
|
866 |
Requesting a user-id from the pool: |
|
867 |
|
|
868 |
:: |
|
869 |
|
|
870 |
from ganeti import ssconf |
|
871 |
from ganeti import uidpool |
|
872 |
|
|
873 |
# Get list of all user-ids in the uid-pool from ssconf |
|
874 |
ss = ssconf.SimpleStore() |
|
875 |
uid_pool = uidpool.ParseUidPool(ss.GetUidPool(), separator="\n") |
|
876 |
all_uids = set(uidpool.ExpandUidPool(uid_pool)) |
|
877 |
|
|
878 |
uid = uidpool.RequestUnusedUid(all_uids) |
|
879 |
try: |
|
880 |
<start a process with the UID> |
|
881 |
# Once the process is started, we can release the file lock |
|
882 |
uid.Unlock() |
|
883 |
except ..., err: |
|
884 |
# Return the UID to the pool |
|
885 |
uidpool.ReleaseUid(uid) |
|
886 |
|
|
887 |
|
|
888 |
Releasing a user-id: |
|
889 |
|
|
890 |
:: |
|
891 |
|
|
892 |
from ganeti import uidpool |
|
893 |
|
|
894 |
uid = <get the UID the process is running under> |
|
895 |
<stop the process> |
|
896 |
uidpool.ReleaseUid(uid) |
|
897 |
|
|
898 |
|
|
728 | 899 |
External interface changes |
729 | 900 |
-------------------------- |
730 | 901 |
|
Also available in: Unified diff