root / docs / design / resource-pool-projects.rst @ 4398adc9
History | View | Annotate | Download (19.6 kB)
1 | f68199bd | Giorgos Korfiatis | Resource-pool projects |
---|---|---|---|
2 | f68199bd | Giorgos Korfiatis | ^^^^^^^^^^^^^^^^^^^^^^ |
3 | f68199bd | Giorgos Korfiatis | |
4 | f68199bd | Giorgos Korfiatis | This document describes the current state of the quota and projects system, |
5 | f68199bd | Giorgos Korfiatis | and proposes a new design for projects that would function as resource |
6 | f68199bd | Giorgos Korfiatis | pools. It sketches implementation details and migration concerns. |
7 | f68199bd | Giorgos Korfiatis | |
8 | f68199bd | Giorgos Korfiatis | Current state and shortcomings |
9 | f68199bd | Giorgos Korfiatis | ============================== |
10 | f68199bd | Giorgos Korfiatis | |
11 | f68199bd | Giorgos Korfiatis | Each Synnefo user is granted quota for several resources. These quota |
12 | f68199bd | Giorgos Korfiatis | originate from two different sources: the system and projects. By default |
13 | f68199bd | Giorgos Korfiatis | a user holds so-called base quota granted by the system upon activation; |
14 | f68199bd | Giorgos Korfiatis | base quota can be customized per user. When a user joins a project, |
15 | f68199bd | Giorgos Korfiatis | resources offered by the project add up to the existing quota, increasing |
16 | f68199bd | Giorgos Korfiatis | the total amount of resources one can reserve. |
17 | f68199bd | Giorgos Korfiatis | |
18 | f68199bd | Giorgos Korfiatis | This design fails to associate an actual (reserved) resource (e.g. VM) with |
19 | f68199bd | Giorgos Korfiatis | a particular project. There is no way to tell which project a resource |
20 | f68199bd | Giorgos Korfiatis | originates from and is thus not possible to employ any targeted policy when |
21 | f68199bd | Giorgos Korfiatis | a user leaves a project, such as reclaiming the granted resource. It is also |
22 | f68199bd | Giorgos Korfiatis | not possible to employ more advanced access control on resources, such as |
23 | f68199bd | Giorgos Korfiatis | sharing VMs among members of a project. |
24 | f68199bd | Giorgos Korfiatis | |
25 | f68199bd | Giorgos Korfiatis | Proposed changes |
26 | f68199bd | Giorgos Korfiatis | ================ |
27 | f68199bd | Giorgos Korfiatis | |
28 | f68199bd | Giorgos Korfiatis | We will alter project semantics so that a project is viewed as a pool of |
29 | f68199bd | Giorgos Korfiatis | finite resources. Each project member can reserve a portion of these |
30 | f68199bd | Giorgos Korfiatis | resources up to a specified limit. Each actual resource (e.g. VM) is |
31 | f68199bd | Giorgos Korfiatis | associated with a particular project. Admission of a user to a project will |
32 | f68199bd | Giorgos Korfiatis | no more result in increasing the user's existing overall quota, but in |
33 | f68199bd | Giorgos Korfiatis | defining new project-specific quota for the user. |
34 | f68199bd | Giorgos Korfiatis | |
35 | f68199bd | Giorgos Korfiatis | A project defines a pair of limits for each resource that it grants (e.g. |
36 | f68199bd | Giorgos Korfiatis | cyclades.vm): project-level limit and member-level limit; The former is the |
37 | f68199bd | Giorgos Korfiatis | total amount of a resource that this project can grant; the latter is the |
38 | f68199bd | Giorgos Korfiatis | maximum amount that an individual user (project member) can reserve and |
39 | f68199bd | Giorgos Korfiatis | cannot exceed the former. A limit on the number of members allowed is still |
40 | f68199bd | Giorgos Korfiatis | enforced. |
41 | f68199bd | Giorgos Korfiatis | |
42 | f68199bd | Giorgos Korfiatis | Projects will be the sole source of resources. Current base quota offered to |
43 | f68199bd | Giorgos Korfiatis | users by the system will be expressed in terms of special-purpose *base* |
44 | 4398adc9 | Giorgos Korfiatis | projects. Due to the central role that projects now acquire, we will alter |
45 | 4398adc9 | Giorgos Korfiatis | the project schema to facilitate project creation and modification without |
46 | 4398adc9 | Giorgos Korfiatis | the extra overhead of submitting and approving applications. |
47 | f68199bd | Giorgos Korfiatis | |
48 | f68199bd | Giorgos Korfiatis | Implementation details |
49 | f68199bd | Giorgos Korfiatis | ====================== |
50 | f68199bd | Giorgos Korfiatis | |
51 | f68199bd | Giorgos Korfiatis | Project-related quota holdings |
52 | f68199bd | Giorgos Korfiatis | ------------------------------ |
53 | f68199bd | Giorgos Korfiatis | |
54 | f68199bd | Giorgos Korfiatis | The Quotaholder is responsible to record all resource allocations and |
55 | f68199bd | Giorgos Korfiatis | deallocations, and enforce the limits. It keeps counters of the following |
56 | f68199bd | Giorgos Korfiatis | structure: |
57 | f68199bd | Giorgos Korfiatis | * resource: the resource name (e.g. cyclades.vm) |
58 | f68199bd | Giorgos Korfiatis | * holder: the entity holding the resource (user or project) |
59 | f68199bd | Giorgos Korfiatis | * source: the origin of the resource; a user-holder reserves from a |
60 | f68199bd | Giorgos Korfiatis | project, a project is a top-level entity and reserves from nowhere (None) |
61 | f68199bd | Giorgos Korfiatis | * limit: maximum allowed allocation (an integer) |
62 | f68199bd | Giorgos Korfiatis | * usage: current allocation (an integer) |
63 | f68199bd | Giorgos Korfiatis | |
64 | f68199bd | Giorgos Korfiatis | [Due to the transactional nature of the mechanism, there are actually two |
65 | f68199bd | Giorgos Korfiatis | usage fields (usage_min and usage_max). Details are beyond the scope of |
66 | f68199bd | Giorgos Korfiatis | this document.] |
67 | f68199bd | Giorgos Korfiatis | |
68 | f68199bd | Giorgos Korfiatis | Creation of a new project triggers the creation of counters like:: |
69 | f68199bd | Giorgos Korfiatis | |
70 | f68199bd | Giorgos Korfiatis | resource holder source limit usage |
71 | f68199bd | Giorgos Korfiatis | ------------|-------------------|--------|-------|------ |
72 | f68199bd | Giorgos Korfiatis | cyclades.vm project:projectID None 50 0 |
73 | f68199bd | Giorgos Korfiatis | |
74 | f68199bd | Giorgos Korfiatis | When a user is admitted in a project, counters are created like:: |
75 | f68199bd | Giorgos Korfiatis | |
76 | f68199bd | Giorgos Korfiatis | resource holder source limit usage |
77 | f68199bd | Giorgos Korfiatis | ------------|---------------|-------------------|-------|------ |
78 | f68199bd | Giorgos Korfiatis | cyclades.vm user:userUUID project:ProjectID 5 0 |
79 | f68199bd | Giorgos Korfiatis | |
80 | f68199bd | Giorgos Korfiatis | Note that the two types of holders (and sources) are made distinguishable with |
81 | f68199bd | Giorgos Korfiatis | a prefix: ``user:`` or ``project:``. |
82 | f68199bd | Giorgos Korfiatis | |
83 | f68199bd | Giorgos Korfiatis | When a user leaves a project, the latter limit is set to zero. This results |
84 | f68199bd | Giorgos Korfiatis | in the project-specific user quota being over limit and prohibits any |
85 | f68199bd | Giorgos Korfiatis | further allocation that would increase this counter. When a project |
86 | f68199bd | Giorgos Korfiatis | is deactivated, the limit of both types of counters is set to zero. |
87 | f68199bd | Giorgos Korfiatis | No user can perform any allocation related to this project. However, the |
88 | f68199bd | Giorgos Korfiatis | holdings cannot be deleted as long as a non-zero usage is recorded. |
89 | f68199bd | Giorgos Korfiatis | Deallocation is always allowed as long as usage does not fall below zero. |
90 | f68199bd | Giorgos Korfiatis | Counters with zero usage and limit could by garbage collected by Astakos, if |
91 | f68199bd | Giorgos Korfiatis | needed. |
92 | f68199bd | Giorgos Korfiatis | |
93 | f68199bd | Giorgos Korfiatis | Base projects |
94 | f68199bd | Giorgos Korfiatis | ------------- |
95 | f68199bd | Giorgos Korfiatis | |
96 | f68199bd | Giorgos Korfiatis | For reasons of uniformity, we replace the base quota mechanism with projects. |
97 | f68199bd | Giorgos Korfiatis | In a similar vein to OpenStack tenants, we define new user-specific *base* |
98 | f68199bd | Giorgos Korfiatis | projects to account for the base quota for each user. These projects should |
99 | f68199bd | Giorgos Korfiatis | be clearly associated with a single user, restrict join/leave actions and |
100 | f68199bd | Giorgos Korfiatis | specify the quota granted by the system. When a new user is created, |
101 | f68199bd | Giorgos Korfiatis | their base project will be automatically created and linked back to the user. |
102 | f68199bd | Giorgos Korfiatis | User activation will trigger project activation, granting the default resource |
103 | f68199bd | Giorgos Korfiatis | quota. Base projects will have no owner, marked thusly as `system' projects. |
104 | f68199bd | Giorgos Korfiatis | The administrator can, following the usual project logic, alter quota by |
105 | f68199bd | Giorgos Korfiatis | modifying the project. Users cannot apply for modification of their base |
106 | f68199bd | Giorgos Korfiatis | projects. |
107 | f68199bd | Giorgos Korfiatis | |
108 | f68199bd | Giorgos Korfiatis | Projects will, from now on, be identified by a UUID. Base projects will |
109 | f68199bd | Giorgos Korfiatis | receive the same UUID as the user itself. ProjectID, which appears above in |
110 | f68199bd | Giorgos Korfiatis | the Quotaholder entries, refers to the project UUID. |
111 | f68199bd | Giorgos Korfiatis | |
112 | f68199bd | Giorgos Korfiatis | Base quota will be expressed both in terms of a project-level and a |
113 | f68199bd | Giorgos Korfiatis | member-level limit. This will result in two operationally equivalent |
114 | f68199bd | Giorgos Korfiatis | Quotaholder counters, as in the following example. In the future, we could |
115 | f68199bd | Giorgos Korfiatis | admit third-party users to a user's base project; in that case, those |
116 | f68199bd | Giorgos Korfiatis | counters would differ. |
117 | f68199bd | Giorgos Korfiatis | |
118 | f68199bd | Giorgos Korfiatis | :: |
119 | f68199bd | Giorgos Korfiatis | |
120 | f68199bd | Giorgos Korfiatis | resource holder source limit usage |
121 | f68199bd | Giorgos Korfiatis | ------------|--------------|--------------|-------|------ |
122 | f68199bd | Giorgos Korfiatis | cyclades.vm project:uuid None 5 1 |
123 | f68199bd | Giorgos Korfiatis | cyclades.vm user:uuid project:uuid 5 1 |
124 | f68199bd | Giorgos Korfiatis | |
125 | 4398adc9 | Giorgos Korfiatis | Private projects |
126 | 4398adc9 | Giorgos Korfiatis | ---------------- |
127 | 4398adc9 | Giorgos Korfiatis | |
128 | 4398adc9 | Giorgos Korfiatis | Since the introduction of base projects will explode the number of total |
129 | 4398adc9 | Giorgos Korfiatis | projects, we will need to control their visibility. We add a new flag |
130 | 4398adc9 | Giorgos Korfiatis | *private* in project definitions. A private project can only be accessed by |
131 | 4398adc9 | Giorgos Korfiatis | its owner and members and not be advertized in the UI. Base projects are |
132 | 4398adc9 | Giorgos Korfiatis | marked as private. |
133 | 4398adc9 | Giorgos Korfiatis | |
134 | 4398adc9 | Giorgos Korfiatis | Decouple projects from applications |
135 | 4398adc9 | Giorgos Korfiatis | ----------------------------------- |
136 | 4398adc9 | Giorgos Korfiatis | |
137 | 4398adc9 | Giorgos Korfiatis | Base projects do not fit well in the current project/application scheme, |
138 | 4398adc9 | Giorgos Korfiatis | because no user has applied for them. Moveover, we would like to easily |
139 | 4398adc9 | Giorgos Korfiatis | modify project properties, particularly quota limits, without the need to |
140 | 4398adc9 | Giorgos Korfiatis | apply for an application for each project and then approve it. |
141 | 4398adc9 | Giorgos Korfiatis | |
142 | 4398adc9 | Giorgos Korfiatis | We will decouple projects from applications by incorporating the project |
143 | 4398adc9 | Giorgos Korfiatis | definition into the project object rather than relying on an application. |
144 | 4398adc9 | Giorgos Korfiatis | The system will directly make a new (base) project upon user creation and a |
145 | 4398adc9 | Giorgos Korfiatis | privileged user will be able to modify an existing project by directly |
146 | 4398adc9 | Giorgos Korfiatis | modifying it. An unprivileged user will still need to make an application. |
147 | 4398adc9 | Giorgos Korfiatis | |
148 | 4398adc9 | Giorgos Korfiatis | The project model is adapted to reference the *last* application that is |
149 | 4398adc9 | Giorgos Korfiatis | related to the project, if any---projects automatically created by the |
150 | 4398adc9 | Giorgos Korfiatis | system reference no application. For an uninitialized project, this |
151 | 4398adc9 | Giorgos Korfiatis | denotes the original application through which the project was made. If |
152 | 4398adc9 | Giorgos Korfiatis | the application is denied or cancelled, the whole project is considered |
153 | 4398adc9 | Giorgos Korfiatis | deleted. |
154 | 4398adc9 | Giorgos Korfiatis | |
155 | 4398adc9 | Giorgos Korfiatis | Applications as modifications |
156 | 4398adc9 | Giorgos Korfiatis | ````````````````````````````` |
157 | 4398adc9 | Giorgos Korfiatis | |
158 | 4398adc9 | Giorgos Korfiatis | Application for a new project is created in state ``pending`` and its |
159 | 4398adc9 | Giorgos Korfiatis | properties are copied into a new project object, which is in state |
160 | 4398adc9 | Giorgos Korfiatis | ``uninitialized``. To preserve this equality, we disallow modifications of |
161 | 4398adc9 | Giorgos Korfiatis | uninitialized projects, either in-place or through an application. An |
162 | 4398adc9 | Giorgos Korfiatis | already activated project can be modified by submitting an application |
163 | 4398adc9 | Giorgos Korfiatis | containing just the desired changes. An application object stores the |
164 | 4398adc9 | Giorgos Korfiatis | specified changes and should remain read-only. |
165 | 4398adc9 | Giorgos Korfiatis | |
166 | 4398adc9 | Giorgos Korfiatis | System default quota and resource registration |
167 | 4398adc9 | Giorgos Korfiatis | ---------------------------------------------- |
168 | f68199bd | Giorgos Korfiatis | |
169 | f68199bd | Giorgos Korfiatis | Each resource registered in the system is assigned a default quota limit. |
170 | 111adca0 | Giorgos Korfiatis | A newly-activated user is given these limits as their base quota. This is |
171 | 111adca0 | Giorgos Korfiatis | till now done by copying the default limits as user's entries in |
172 | 111adca0 | Giorgos Korfiatis | AstakosUserQuota. Default limits will from now on be copied into the base |
173 | 111adca0 | Giorgos Korfiatis | project's resource definitions. |
174 | 111adca0 | Giorgos Korfiatis | |
175 | 111adca0 | Giorgos Korfiatis | Conventional projects are created through a project application, which |
176 | 111adca0 | Giorgos Korfiatis | may not specify limits for all resources registered in the system. In |
177 | 111adca0 | Giorgos Korfiatis | fact, it may even be impossible to specify a resource, if it is set |
178 | 111adca0 | Giorgos Korfiatis | ``api_visible=False``. We have to somehow specify these limits. Defaulting |
179 | 111adca0 | Giorgos Korfiatis | to zero is not appropriate: if we don't want to control a resource, we |
180 | 111adca0 | Giorgos Korfiatis | would like it set to infinite. We thus need an extra skeleton, like the |
181 | 111adca0 | Giorgos Korfiatis | one specifying the default base quota, in order to fill in missing limits |
182 | 111adca0 | Giorgos Korfiatis | for conventional projects. It will be controled by a new option |
183 | 111adca0 | Giorgos Korfiatis | ``--project-default`` of command ``resource-modify``. |
184 | f68199bd | Giorgos Korfiatis | |
185 | 4398adc9 | Giorgos Korfiatis | When a project is activated, either directly in the case of base projects |
186 | 4398adc9 | Giorgos Korfiatis | or through the approval of a project application, limits for resources not |
187 | 4398adc9 | Giorgos Korfiatis | specified are automatically completed by consulting the appropriate |
188 | 4398adc9 | Giorgos Korfiatis | skeleton. |
189 | f68199bd | Giorgos Korfiatis | |
190 | f68199bd | Giorgos Korfiatis | Allocation of a new resource |
191 | f68199bd | Giorgos Korfiatis | ---------------------------- |
192 | f68199bd | Giorgos Korfiatis | |
193 | f68199bd | Giorgos Korfiatis | When a service allocates a new resource, it should associate it both with a |
194 | f68199bd | Giorgos Korfiatis | user and a project. The commission issued to the Quotaholder should attempt |
195 | f68199bd | Giorgos Korfiatis | to update all related counters. For example, it should include the following |
196 | f68199bd | Giorgos Korfiatis | provisions:: |
197 | f68199bd | Giorgos Korfiatis | |
198 | f68199bd | Giorgos Korfiatis | "provisions": [ |
199 | f68199bd | Giorgos Korfiatis | { |
200 | f68199bd | Giorgos Korfiatis | "holder": "user:user-uuid", |
201 | f68199bd | Giorgos Korfiatis | "source": "project:project-uuid", |
202 | f68199bd | Giorgos Korfiatis | "resource": "cyclades.vm", |
203 | f68199bd | Giorgos Korfiatis | "quantity": 1 |
204 | f68199bd | Giorgos Korfiatis | }, |
205 | f68199bd | Giorgos Korfiatis | { |
206 | f68199bd | Giorgos Korfiatis | "holder": "project:project-uuid", |
207 | f68199bd | Giorgos Korfiatis | "source": None, |
208 | f68199bd | Giorgos Korfiatis | "resource": "cyclades.vm", |
209 | f68199bd | Giorgos Korfiatis | "quantity": 1 |
210 | f68199bd | Giorgos Korfiatis | }, |
211 | f68199bd | Giorgos Korfiatis | { |
212 | f68199bd | Giorgos Korfiatis | "holder": "user:user-uuid", |
213 | f68199bd | Giorgos Korfiatis | "source": "project:project-uuid", |
214 | f68199bd | Giorgos Korfiatis | "resource": "cyclades.cpu", |
215 | f68199bd | Giorgos Korfiatis | "quantity": 2 |
216 | f68199bd | Giorgos Korfiatis | }, |
217 | f68199bd | Giorgos Korfiatis | { |
218 | f68199bd | Giorgos Korfiatis | "holder": "project:project-uuid", |
219 | f68199bd | Giorgos Korfiatis | "source": None, |
220 | f68199bd | Giorgos Korfiatis | "resource": "cyclades.cpu", |
221 | f68199bd | Giorgos Korfiatis | "quantity": 2 |
222 | f68199bd | Giorgos Korfiatis | } |
223 | f68199bd | Giorgos Korfiatis | ] |
224 | f68199bd | Giorgos Korfiatis | |
225 | f68199bd | Giorgos Korfiatis | If any of these provisions fails, i.e. either on the project-level limits or |
226 | f68199bd | Giorgos Korfiatis | the user-level ones, the whole commission fails. |
227 | f68199bd | Giorgos Korfiatis | |
228 | f68199bd | Giorgos Korfiatis | The astakosclient call ``issue_one_commission`` will be adapted to abstract |
229 | f68199bd | Giorgos Korfiatis | away the need to write both the user-level and the project-level provisions. |
230 | f68199bd | Giorgos Korfiatis | The previous commission will be issued with:: |
231 | f68199bd | Giorgos Korfiatis | |
232 | 9b94cf0f | Giorgos Korfiatis | issue_one_commission(holder="user-uuid", source="project-uuid", |
233 | f68199bd | Giorgos Korfiatis | provisions={"cyclades.vm": 1, "cyclades.cpu": 2}) |
234 | f68199bd | Giorgos Korfiatis | |
235 | f68199bd | Giorgos Korfiatis | The service is responsible to record this resource-to-project association. |
236 | f68199bd | Giorgos Korfiatis | In Cyclades, each VM, floating IP, or other distinct resource should be |
237 | f68199bd | Giorgos Korfiatis | linked to a project. Pithos should link containers to projects. |
238 | f68199bd | Giorgos Korfiatis | |
239 | f68199bd | Giorgos Korfiatis | Astakos will handle its own resource ``astakos.pending_app`` in a special |
240 | 4398adc9 | Giorgos Korfiatis | way: it will always be charged at the user's base project. |
241 | f68199bd | Giorgos Korfiatis | |
242 | f68199bd | Giorgos Korfiatis | Resource reassignment |
243 | f68199bd | Giorgos Korfiatis | --------------------- |
244 | f68199bd | Giorgos Korfiatis | |
245 | f68199bd | Giorgos Korfiatis | The system will support reassigning a resource to a new project. One needs |
246 | f68199bd | Giorgos Korfiatis | to specify all related resource values. Astakosclient will provide a |
247 | 9b94cf0f | Giorgos Korfiatis | convenience function ``issue_resource_reassignment`` to construct all needed |
248 | f68199bd | Giorgos Korfiatis | provisions. For instance, reassigning a VM with two CPUs can be done with:: |
249 | f68199bd | Giorgos Korfiatis | |
250 | 9b94cf0f | Giorgos Korfiatis | issue_resource_reassignment(holder="user-uuid", |
251 | 9b94cf0f | Giorgos Korfiatis | from_source="from-uuid", to_source="to-uuid", |
252 | 9b94cf0f | Giorgos Korfiatis | provisions={"cyclades.vm": 1, "cyclades.cpu": 2}) |
253 | f68199bd | Giorgos Korfiatis | |
254 | f68199bd | Giorgos Korfiatis | This will issue the following provisions to the Quotaholder:: |
255 | f68199bd | Giorgos Korfiatis | |
256 | f68199bd | Giorgos Korfiatis | "provisions": [ |
257 | f68199bd | Giorgos Korfiatis | { |
258 | f68199bd | Giorgos Korfiatis | "holder": "user:user-uuid", |
259 | f68199bd | Giorgos Korfiatis | "source": "project:from-uuid", |
260 | f68199bd | Giorgos Korfiatis | "resource": "cyclades.vm", |
261 | f68199bd | Giorgos Korfiatis | "quantity": -1 |
262 | f68199bd | Giorgos Korfiatis | }, |
263 | f68199bd | Giorgos Korfiatis | { |
264 | f68199bd | Giorgos Korfiatis | "holder": "project:from-uuid", |
265 | f68199bd | Giorgos Korfiatis | "source": None, |
266 | f68199bd | Giorgos Korfiatis | "resource": "cyclades.vm", |
267 | f68199bd | Giorgos Korfiatis | "quantity": -1 |
268 | f68199bd | Giorgos Korfiatis | }, |
269 | f68199bd | Giorgos Korfiatis | { |
270 | f68199bd | Giorgos Korfiatis | "holder": "user:user-uuid", |
271 | f68199bd | Giorgos Korfiatis | "source": "project:from-uuid", |
272 | f68199bd | Giorgos Korfiatis | "resource": "cyclades.cpu", |
273 | f68199bd | Giorgos Korfiatis | "quantity": -2 |
274 | f68199bd | Giorgos Korfiatis | }, |
275 | f68199bd | Giorgos Korfiatis | { |
276 | f68199bd | Giorgos Korfiatis | "holder": "project:from-uuid", |
277 | f68199bd | Giorgos Korfiatis | "source": None, |
278 | f68199bd | Giorgos Korfiatis | "resource": "cyclades.cpu", |
279 | f68199bd | Giorgos Korfiatis | "quantity": -2 |
280 | f68199bd | Giorgos Korfiatis | }, |
281 | f68199bd | Giorgos Korfiatis | { |
282 | f68199bd | Giorgos Korfiatis | "holder": "user:user-uuid", |
283 | f68199bd | Giorgos Korfiatis | "source": "project:to-uuid", |
284 | f68199bd | Giorgos Korfiatis | "resource": "cyclades.vm", |
285 | f68199bd | Giorgos Korfiatis | "quantity": 1 |
286 | f68199bd | Giorgos Korfiatis | }, |
287 | f68199bd | Giorgos Korfiatis | { |
288 | f68199bd | Giorgos Korfiatis | "holder": "project:to-uuid", |
289 | f68199bd | Giorgos Korfiatis | "source": None, |
290 | f68199bd | Giorgos Korfiatis | "resource": "cyclades.vm", |
291 | f68199bd | Giorgos Korfiatis | "quantity": 1 |
292 | f68199bd | Giorgos Korfiatis | } |
293 | f68199bd | Giorgos Korfiatis | { |
294 | f68199bd | Giorgos Korfiatis | "holder": "user:user-uuid", |
295 | f68199bd | Giorgos Korfiatis | "source": "project:to-uuid", |
296 | f68199bd | Giorgos Korfiatis | "resource": "cyclades.cpu", |
297 | f68199bd | Giorgos Korfiatis | "quantity": 2 |
298 | f68199bd | Giorgos Korfiatis | }, |
299 | f68199bd | Giorgos Korfiatis | { |
300 | f68199bd | Giorgos Korfiatis | "holder": "project:to-uuid", |
301 | f68199bd | Giorgos Korfiatis | "source": None, |
302 | f68199bd | Giorgos Korfiatis | "resource": "cyclades.cpu", |
303 | f68199bd | Giorgos Korfiatis | "quantity": 2 |
304 | f68199bd | Giorgos Korfiatis | } |
305 | f68199bd | Giorgos Korfiatis | ] |
306 | f68199bd | Giorgos Korfiatis | |
307 | 4398adc9 | Giorgos Korfiatis | API changes |
308 | 4398adc9 | Giorgos Korfiatis | ----------- |
309 | f68199bd | Giorgos Korfiatis | |
310 | f68199bd | Giorgos Korfiatis | API call ``GET /quotas`` is extended to incorporate project-level quota. The |
311 | f68199bd | Giorgos Korfiatis | response contains entries for all projects for which a user/project pair |
312 | f68199bd | Giorgos Korfiatis | exists in the quotaholder:: |
313 | f68199bd | Giorgos Korfiatis | |
314 | f68199bd | Giorgos Korfiatis | { |
315 | f68199bd | Giorgos Korfiatis | "project1-uuid": { |
316 | f68199bd | Giorgos Korfiatis | "cyclades.ram": { |
317 | f68199bd | Giorgos Korfiatis | "usage": 2147483648, |
318 | f68199bd | Giorgos Korfiatis | "limit": 2147483648, |
319 | f68199bd | Giorgos Korfiatis | "pending": 0, |
320 | f68199bd | Giorgos Korfiatis | "project_usage": ..., |
321 | f68199bd | Giorgos Korfiatis | "project_limit": ..., |
322 | f68199bd | Giorgos Korfiatis | "project_pending": ... |
323 | f68199bd | Giorgos Korfiatis | }, |
324 | f68199bd | Giorgos Korfiatis | "cyclades.vm": { |
325 | f68199bd | Giorgos Korfiatis | ... |
326 | f68199bd | Giorgos Korfiatis | } |
327 | f68199bd | Giorgos Korfiatis | } |
328 | f68199bd | Giorgos Korfiatis | "project2-uuid": { |
329 | f68199bd | Giorgos Korfiatis | ... |
330 | f68199bd | Giorgos Korfiatis | } |
331 | f68199bd | Giorgos Korfiatis | } |
332 | f68199bd | Giorgos Korfiatis | |
333 | f68199bd | Giorgos Korfiatis | An extra or differentiated call may be needed to retrieve the project quota |
334 | f68199bd | Giorgos Korfiatis | regardless of user:: |
335 | f68199bd | Giorgos Korfiatis | |
336 | f68199bd | Giorgos Korfiatis | GET /quotas?mode=projects |
337 | f68199bd | Giorgos Korfiatis | |
338 | f68199bd | Giorgos Korfiatis | { |
339 | f68199bd | Giorgos Korfiatis | "project-uuid": { |
340 | f68199bd | Giorgos Korfiatis | "cyclades.ram": { |
341 | f68199bd | Giorgos Korfiatis | "project_usage": 2147483648, |
342 | f68199bd | Giorgos Korfiatis | "project_limit": 2147483648, |
343 | f68199bd | Giorgos Korfiatis | "project_pending": 0 |
344 | f68199bd | Giorgos Korfiatis | } |
345 | f68199bd | Giorgos Korfiatis | "cyclades.vm": { |
346 | f68199bd | Giorgos Korfiatis | ... |
347 | f68199bd | Giorgos Korfiatis | } |
348 | f68199bd | Giorgos Korfiatis | } |
349 | f68199bd | Giorgos Korfiatis | } |
350 | f68199bd | Giorgos Korfiatis | |
351 | 4398adc9 | Giorgos Korfiatis | ``GET /service_project_quotas`` will be used in a similar way as ``GET |
352 | 4398adc9 | Giorgos Korfiatis | /service_quotas`` to get the project-level quotas for resources associated |
353 | 4398adc9 | Giorgos Korfiatis | with the Synnefo component that makes the request. |
354 | 4398adc9 | Giorgos Korfiatis | |
355 | f68199bd | Giorgos Korfiatis | All service API calls that create resources can specify the project where |
356 | 9b94cf0f | Giorgos Korfiatis | they will be attributed. |
357 | 9b94cf0f | Giorgos Korfiatis | |
358 | 9b94cf0f | Giorgos Korfiatis | In cyclades, ``POST /servers`` (likewise for networks and floating IPs) will |
359 | 9b94cf0f | Giorgos Korfiatis | receive an extra argument ``project``. If it is missing, the user's base |
360 | 9b94cf0f | Giorgos Korfiatis | project will be assumed. In calls detailing a resource (e.g., ``GET |
361 | 9b94cf0f | Giorgos Korfiatis | /servers/<server_id>``), the field ``tenant_id`` will contain the |
362 | 9b94cf0f | Giorgos Korfiatis | project id. |
363 | 9b94cf0f | Giorgos Korfiatis | |
364 | 9b94cf0f | Giorgos Korfiatis | Moreover, extra calls will be needed for resource reassignment, |
365 | f68199bd | Giorgos Korfiatis | e.g:: |
366 | f68199bd | Giorgos Korfiatis | |
367 | f68199bd | Giorgos Korfiatis | POST /servers/<server-id>/action |
368 | f68199bd | Giorgos Korfiatis | |
369 | f68199bd | Giorgos Korfiatis | { |
370 | 9b94cf0f | Giorgos Korfiatis | "reassign": {"project": <project-id>} |
371 | f68199bd | Giorgos Korfiatis | } |
372 | f68199bd | Giorgos Korfiatis | |
373 | 9b94cf0f | Giorgos Korfiatis | In pithos, ``PUT`` and ``POST`` calls at the container level will accept an |
374 | 9b94cf0f | Giorgos Korfiatis | extra optional policy ``project``. The former call assigns a newly created |
375 | 9b94cf0f | Giorgos Korfiatis | container to a given project, the latter reassigns an existing container. |
376 | 9b94cf0f | Giorgos Korfiatis | Field ``x-container-policy-project`` will be retrieved by a ``HEAD`` call at |
377 | 9b94cf0f | Giorgos Korfiatis | the container level. |
378 | 9b94cf0f | Giorgos Korfiatis | |
379 | 4398adc9 | Giorgos Korfiatis | Changes in the projects API |
380 | 4398adc9 | Giorgos Korfiatis | ``````````````````````````` |
381 | 4398adc9 | Giorgos Korfiatis | |
382 | 4398adc9 | Giorgos Korfiatis | ``PUT /projects`` will be used to make a new project replacing ``POST``. |
383 | 4398adc9 | Giorgos Korfiatis | |
384 | 4398adc9 | Giorgos Korfiatis | ``POST /projects/<proj_id>`` now expects a dictionary with just the desired |
385 | 4398adc9 | Giorgos Korfiatis | changes, not a complete project definition. It is only allowed if the |
386 | 4398adc9 | Giorgos Korfiatis | project is already activated. |
387 | 4398adc9 | Giorgos Korfiatis | |
388 | 4398adc9 | Giorgos Korfiatis | ``GET /projects/<proj_id>`` changes to include a ``last_application`` field, |
389 | 4398adc9 | Giorgos Korfiatis | if applicable. |
390 | 4398adc9 | Giorgos Korfiatis | |
391 | 4398adc9 | Giorgos Korfiatis | Application actions (approve, deny, dismiss, cancel) are integrated into |
392 | 4398adc9 | Giorgos Korfiatis | project actions and expect an extra ``app_id`` argument to specify the |
393 | 4398adc9 | Giorgos Korfiatis | application. Actions are allowed only on a project's last application; |
394 | 4398adc9 | Giorgos Korfiatis | the application id is required in order to avoid races. |
395 | 4398adc9 | Giorgos Korfiatis | |
396 | 4398adc9 | Giorgos Korfiatis | The applications API is removed, incorporated into the projects API. |
397 | 4398adc9 | Giorgos Korfiatis | |
398 | f68199bd | Giorgos Korfiatis | User interface |
399 | f68199bd | Giorgos Korfiatis | -------------- |
400 | f68199bd | Giorgos Korfiatis | |
401 | f68199bd | Giorgos Korfiatis | User quota will be presented per project, including the aggregate activity |
402 | f68199bd | Giorgos Korfiatis | of other project members: the Resource Usage page will include a drop-down |
403 | f68199bd | Giorgos Korfiatis | menu with all relevant projects. By default, user's base project will |
404 | f68199bd | Giorgos Korfiatis | be assumed. When choosing a project, usage for all resources will be |
405 | f68199bd | Giorgos Korfiatis | presented for the given project in the following style:: |
406 | f68199bd | Giorgos Korfiatis | |
407 | f68199bd | Giorgos Korfiatis | limit |
408 | f68199bd | Giorgos Korfiatis | used ^ taken by others |
409 | f68199bd | Giorgos Korfiatis | |::::::|..............|...........|::::::::::::::::::::::::::::::::::| |
410 | f68199bd | Giorgos Korfiatis | ^ ^ ^ |
411 | f68199bd | Giorgos Korfiatis | usage effective project |
412 | f68199bd | Giorgos Korfiatis | limit limit |
413 | f68199bd | Giorgos Korfiatis | |
414 | f68199bd | Giorgos Korfiatis | |
415 | f68199bd | Giorgos Korfiatis | limit |
416 | f68199bd | Giorgos Korfiatis | used ^ taken by others |
417 | f68199bd | Giorgos Korfiatis | |::::::|........|:::::|::::::::::::::::::::::::::::::::::::::::::::::| |
418 | f68199bd | Giorgos Korfiatis | ^ ^ ^ |
419 | f68199bd | Giorgos Korfiatis | usage effective project |
420 | f68199bd | Giorgos Korfiatis | limit limit |
421 | f68199bd | Giorgos Korfiatis | |
422 | f68199bd | Giorgos Korfiatis | Text accompanying the bar could mention usage based on the effective limit, |
423 | f68199bd | Giorgos Korfiatis | e.g.: `usage` out of `effective limit` Virtual Machines. Likewise the shaded |
424 | f68199bd | Giorgos Korfiatis | `used` part of the bar could express the same ratio in percentage terms. |
425 | f68199bd | Giorgos Korfiatis | |
426 | f68199bd | Giorgos Korfiatis | Given the above-mentioned response of the ``/quotas`` call, the effective |
427 | f68199bd | Giorgos Korfiatis | limit can be computed by:: |
428 | f68199bd | Giorgos Korfiatis | |
429 | f68199bd | Giorgos Korfiatis | taken_by_others = project_usage - usage |
430 | f68199bd | Giorgos Korfiatis | effective_limit = min(limit, project_limit - taken_by_others) |
431 | f68199bd | Giorgos Korfiatis | |
432 | f68199bd | Giorgos Korfiatis | Projects show up in a number of service-specific user interactions, too. |
433 | f68199bd | Giorgos Korfiatis | When creating a Cyclades VM, the flavor-choosing window should first ask |
434 | f68199bd | Giorgos Korfiatis | for the project where the VM will be charged before showing the |
435 | f68199bd | Giorgos Korfiatis | available resource combinations. Likewise, creating a new container in |
436 | f68199bd | Giorgos Korfiatis | Pithos will prompt for picking a project to associate with. |
437 | f68199bd | Giorgos Korfiatis | |
438 | f68199bd | Giorgos Korfiatis | Resource presentation (e.g. Cyclades VMs) will also mention the associated |
439 | f68199bd | Giorgos Korfiatis | project and provide an action to reassign the resource to a different |
440 | f68199bd | Giorgos Korfiatis | project. |
441 | f68199bd | Giorgos Korfiatis | |
442 | f68199bd | Giorgos Korfiatis | Command-line interface |
443 | f68199bd | Giorgos Korfiatis | ---------------------- |
444 | f68199bd | Giorgos Korfiatis | |
445 | f68199bd | Giorgos Korfiatis | Quota can be queried per user or project:: |
446 | f68199bd | Giorgos Korfiatis | |
447 | f68199bd | Giorgos Korfiatis | # snf-manage user-show <id> --quota |
448 | f68199bd | Giorgos Korfiatis | |
449 | f68199bd | Giorgos Korfiatis | project resource limit effective_limit usage |
450 | f68199bd | Giorgos Korfiatis | ------------------------------------------------- |
451 | f68199bd | Giorgos Korfiatis | uuid cyclades.vm 10 9 5 |
452 | f68199bd | Giorgos Korfiatis | |
453 | f68199bd | Giorgos Korfiatis | # snf-manage project-show <id> --quota |
454 | f68199bd | Giorgos Korfiatis | |
455 | f68199bd | Giorgos Korfiatis | resource limit usage |
456 | f68199bd | Giorgos Korfiatis | ------------------------ |
457 | f68199bd | Giorgos Korfiatis | cyclades.vm 100 50 |
458 | f68199bd | Giorgos Korfiatis | |
459 | 4398adc9 | Giorgos Korfiatis | A new command ``snf-manage project-modify`` will enable in-place |
460 | 4398adc9 | Giorgos Korfiatis | modification of project properties, such as their quota limits. |
461 | f68199bd | Giorgos Korfiatis | |
462 | f68199bd | Giorgos Korfiatis | Currently, the administrator can change the user base quota with: |
463 | 45112d5a | Giorgos Korfiatis | ``snf-manage user-modify <id> --base-quota <resource> <capacity>``. |
464 | f68199bd | Giorgos Korfiatis | This will be removed in favor of the ``project-modify`` command, so that all |
465 | 111adca0 | Giorgos Korfiatis | quota are handled in a uniform way. Similar to ``user-modify --all``, |
466 | 4398adc9 | Giorgos Korfiatis | ``project-modify`` will get options ``--all-base-projects`` to |
467 | 4398adc9 | Giorgos Korfiatis | allow updating base quota in bulk. |
468 | f68199bd | Giorgos Korfiatis | |
469 | f68199bd | Giorgos Korfiatis | Migration steps |
470 | f68199bd | Giorgos Korfiatis | =============== |
471 | f68199bd | Giorgos Korfiatis | |
472 | f68199bd | Giorgos Korfiatis | Project conversion |
473 | f68199bd | Giorgos Korfiatis | ------------------ |
474 | f68199bd | Giorgos Korfiatis | |
475 | f68199bd | Giorgos Korfiatis | Existing projects need to be converted to resource-pool ones. The following |
476 | f68199bd | Giorgos Korfiatis | steps must be taken in Astakos: |
477 | f68199bd | Giorgos Korfiatis | * compute project-level limits for each resource as |
478 | f68199bd | Giorgos Korfiatis | max_members * member-level limit |
479 | f68199bd | Giorgos Korfiatis | * create base projects based on base quota for each user |
480 | f68199bd | Giorgos Korfiatis | * make Quotaholder entries for projects and user/project pairs |
481 | f68199bd | Giorgos Korfiatis | * assign all current usage to the base projects (both project |
482 | f68199bd | Giorgos Korfiatis | and user/project entries) |
483 | f68199bd | Giorgos Korfiatis | * set usage for all other entries to zero |
484 | f68199bd | Giorgos Korfiatis | |
485 | f68199bd | Giorgos Korfiatis | Cyclades and Pithos should initialize their project attribute on each resource |
486 | f68199bd | Giorgos Korfiatis | with the user's base project, that is, the same UUID as the resource owner. |
487 | f68199bd | Giorgos Korfiatis | |
488 | f68199bd | Giorgos Korfiatis | Initial resource reassignment |
489 | f68199bd | Giorgos Korfiatis | ----------------------------- |
490 | f68199bd | Giorgos Korfiatis | |
491 | f68199bd | Giorgos Korfiatis | Once migration has finished, users will be off-quota on their base project, |
492 | f68199bd | Giorgos Korfiatis | if they had used additional quota from projects. To alleviate this |
493 | f68199bd | Giorgos Korfiatis | situation, each service can attempt to reassign resources to other projects, |
494 | f68199bd | Giorgos Korfiatis | following this strategy: |
495 | f68199bd | Giorgos Korfiatis | * consult Astakos for projects and quota for a given user |
496 | f68199bd | Giorgos Korfiatis | * select resources that can fit in another project |
497 | f68199bd | Giorgos Korfiatis | * issue a commission to decrease usage of the base project and likewise |
498 | f68199bd | Giorgos Korfiatis | increase usage of the available project |
499 | f68199bd | Giorgos Korfiatis | * record the new ProjectUUID for the reassigned resources |