root / snf-astakos-app / docs / projects_specs.rst @ 075b91de
History | View | Annotate | Download (21.8 kB)
1 |
Synnefo Projects: General Design Notes |
---|---|
2 |
====================================== |
3 |
|
4 |
Projects can be thought of as "contracts" between the infrastructure |
5 |
and the members of the Project, so that members can allocate and use |
6 |
resources for a specified period in time. |
7 |
For every Project there is an *application*, and a *membership set*. |
8 |
The application must be approved before the Project "contract" |
9 |
comes in effect, and before any members join. |
10 |
|
11 |
Applications contain a Project *definition* that formally includes |
12 |
all the policy of the Project "contract", such as name, ownership, |
13 |
dates, and resource limits. |
14 |
Applications and the definitions they contain are, like contracts, |
15 |
immutable objects that can be tracked and effected in their entirety. |
16 |
Every change made to a pending application or an existing Project |
17 |
must be applied for through a new application. |
18 |
|
19 |
Project Synchronization |
20 |
----------------------- |
21 |
A Project has two kinds of effects, global and per-membership. |
22 |
Global effects are encoded within the Project object itself, |
23 |
and may include services (e.g. dns name, website, forum/collaboration), |
24 |
project-wide resources (e.g. VMs, IPs, diskspace), etc. |
25 |
Per-membership effects are encoded within Membership objects, |
26 |
and are prescribed by the policy in the project definition. |
27 |
|
28 |
By design, the effects of the project are expected to extend |
29 |
beyond the database system that records the project approval, |
30 |
therefore these effects cannot be relied to be implemented |
31 |
instantly and atomically upon request. |
32 |
For example, project creation or modification may involve |
33 |
updating resource management services, notification services, |
34 |
project website services, which may all have their own separate |
35 |
database systems. |
36 |
|
37 |
Consequently, once a policy is (atomically) registered as being in effect, |
38 |
it must be propagated for implementation with separate (non-atomic) procedures, |
39 |
collectively called Project Synchronization. |
40 |
|
41 |
Currently, the general approach for Synchronization of synnefo Projects, |
42 |
is to maintain appropriate states for the objects encoding policy effects |
43 |
(i.e. Project and Membership objects), and execute careful transitions |
44 |
among those states, respecting semanting dependencies and limitations, |
45 |
and remote system accesses. |
46 |
For example, a project cannot be considered "synchronized" unless |
47 |
all its Memberships are also considered "synchronized", |
48 |
or you cannot declare a Membership as synchronized if the newly-set |
49 |
quotas have not been acknowledged by the remote quota service. |
50 |
|
51 |
Creating and Modifying Projects |
52 |
------------------------------- |
53 |
Projects are created and modified by issuing a corresponding application. |
54 |
Applications that refer to the same Project are chained by including |
55 |
a reference to the precursor application into the new one. |
56 |
Applications that do not specify a precursor always create a new Project |
57 |
(that is, a new project object with a new membership set). |
58 |
Project applications that have not yet been approved may also be modified. |
59 |
When an unapproved application is modified (ie. it is succeeded by another), |
60 |
it may or may not be automatically rejected, depending on policy. |
61 |
|
62 |
Projects once created, are expected to always remain in record, |
63 |
even if they have been deactivated (i.e. their policies removed from effect). |
64 |
Deactivating a project may be the result of a policy-specific action, |
65 |
such as *termination* on a pre-defined date or *suspension* following |
66 |
an administrative decision. |
67 |
Deactivated projects retain both their Definition and their Membership sets, |
68 |
allowing them to be reactivated by another kind of policy-specific action. |
69 |
|
70 |
Adding and Removing Project Members |
71 |
----------------------------------- |
72 |
Memberships once created, are also expected to remain in historical record, |
73 |
even if they have been marked as removed by a policy-specific action, |
74 |
such as the user leaving the project, |
75 |
or the project owner suspending, or terminating them altogether. |
76 |
|
77 |
|
78 |
|
79 |
Reference Schema for Projects |
80 |
============================= |
81 |
|
82 |
General Terms |
83 |
------------- |
84 |
For clarity and precision, we (try to) use different terms for similar meanings. |
85 |
We also separate low-level technical primitives from related higher-level |
86 |
policy actions that include them. |
87 |
|
88 |
synchronize, synchronized, synchronization |
89 |
A low-level term for Projects and Memberships. |
90 |
Refers to the controlled (non-atomic) process that implements |
91 |
a newly modified policy (i.e. adds or removes policy effects |
92 |
such as quota limits). |
93 |
|
94 |
activate, deactivate, active, inactive |
95 |
A low-level term for Projects and Memberships. |
96 |
A Project or Membership is considered active as long as |
97 |
its policy is in effect. |
98 |
|
99 |
suspend, suspended, suspension |
100 |
A policy term for Projects and Memberships, |
101 |
implying temporary deactivation by an administrative action |
102 |
(e.g. abuse report, limits violation). |
103 |
|
104 |
terminate, terminated, termination |
105 |
A policyu term for Projects and Memberships, |
106 |
implying permanent deactivation, especially |
107 |
according to a pre-defined end-of-life event |
108 |
(e.g. project/contract "expiration"). |
109 |
|
110 |
add or remove membership |
111 |
Low-level terms for managing memberships to projects. |
112 |
|
113 |
join or leave project |
114 |
Policy terms for when users request their addition or removal |
115 |
from a project membership set. |
116 |
|
117 |
accept or reject membership request |
118 |
Policy terms for a project administrator to decide on |
119 |
join or leave requests. |
120 |
|
121 |
approve or disapprove project |
122 |
Policy terms for service administrators to decide whether |
123 |
to create/activate a project according or not |
124 |
|
125 |
alter project |
126 |
Low-level term for changes made to the global (i.e. not Membership) |
127 |
project status (e.g. new application, suspension). |
128 |
|
129 |
modify project |
130 |
Policy term for submitting a new application as a successor |
131 |
to an existing one, to alter the definition of a project. |
132 |
|
133 |
project leader |
134 |
The user who has authority over a project, |
135 |
to accept or reject membership requests, |
136 |
or to perform other actions according to policy. |
137 |
|
138 |
project applicant |
139 |
The user who submits a project application for creation |
140 |
or modification of a project. |
141 |
The applicant can be a different user than the leader. |
142 |
|
143 |
project administrator |
144 |
A user who has authority to approve, disapprove, and modify |
145 |
projects of a certain class (e.g. according to their domain names). |
146 |
|
147 |
|
148 |
Definition |
149 |
---------- |
150 |
A **definition** for a project, which represents the initialization |
151 |
or modification of a project, has the following attributes: |
152 |
|
153 |
``name`` |
154 |
*in dot-hierarchical dns-like form* |
155 |
|
156 |
``description`` |
157 |
*text describing the project for the public* |
158 |
|
159 |
``start_date`` |
160 |
*when the project is requested to become active* |
161 |
|
162 |
``end_date`` |
163 |
*when the project is to be deactivatedended* |
164 |
|
165 |
``member_join_policy`` |
166 |
*an enumeration of policies on how new join requests are to be accepted. |
167 |
The policies include the following:* |
168 |
|
169 |
``auto_accept`` |
170 |
*new join requests are automatically accepted by the system* |
171 |
|
172 |
``owner_accepts`` |
173 |
*new join requests must be accepted by the owner of the project* |
174 |
|
175 |
``closed`` |
176 |
*no new members can join the project, even if old ones leave* |
177 |
|
178 |
``member_leave_policy`` |
179 |
*an enumeration of policies on how new leave requests are to be accepted. |
180 |
The policies include the following:* |
181 |
|
182 |
``auto_accept`` |
183 |
*leave requests are automatically accepted by the system* |
184 |
|
185 |
``owner_accepts`` |
186 |
*leave requests must be accepted by the owner of the project** |
187 |
|
188 |
``closed`` |
189 |
*no member can leave the project* |
190 |
|
191 |
``limit_on_member_count`` |
192 |
*the maximum number of members that can be admitted to the project* |
193 |
|
194 |
``limits_on_resources`` |
195 |
*a set of limits on the total resources the group can grant to its members* |
196 |
|
197 |
``resource_grants`` |
198 |
*a set of resource usage limits each member is granted by the project* |
199 |
|
200 |
|
201 |
|
202 |
Application for a Project |
203 |
------------------------- |
204 |
An **application** for a project must be issued by a user and |
205 |
*approved* by the service before any resources are granted. |
206 |
Its attributes are: |
207 |
|
208 |
``serial`` |
209 |
*a unique serial number identifying the application* |
210 |
|
211 |
``applicant`` |
212 |
*who applies for creation or modification of a project* |
213 |
|
214 |
``owner`` |
215 |
*the requested owner of the project* |
216 |
|
217 |
``comments`` |
218 |
*text comments for the application* |
219 |
|
220 |
``definition`` |
221 |
*the definition of the project being requested* |
222 |
|
223 |
``issue_date`` |
224 |
*when the application was submitted* |
225 |
|
226 |
``precursor_application`` |
227 |
*a reference by serial to a previous application which this application |
228 |
requests to modify. It can be null if there is no precursor.* |
229 |
|
230 |
An application is immutable in its attributes above, |
231 |
but its status depends on how it has been processed. |
232 |
The *application status* can be: |
233 |
|
234 |
:(1): pending |
235 |
:(2a): approved |
236 |
:(2b): rejected |
237 |
:(3): replaced |
238 |
|
239 |
When an application becomes *approved* and therefore defines |
240 |
a the project, its precursor (if any) must atomically be set to *replaced*. |
241 |
|
242 |
|
243 |
|
244 |
Project Membership |
245 |
------------------------- |
246 |
A *project membership* maps a user to a project and holds state for this mapping. |
247 |
There are no inherent constraints to this mapping, |
248 |
any user might be a member to any project. |
249 |
|
250 |
The **state** of membership can be: |
251 |
|
252 |
:(1): *pending acceptance* |
253 |
:(2a): *rejected* |
254 |
:(2b): *accepted, pending synchronization* |
255 |
:(3): *active* |
256 |
:(4): *pending removal* |
257 |
:(5): *removed, pending synchronization* |
258 |
:(6): *removed* |
259 |
|
260 |
The transitions from 2b to 3, and 5 to 6, must first commit |
261 |
their starting state and then only update to the next state |
262 |
after the *Synchronize Membership* procedure has been |
263 |
acknowledged as successful. |
264 |
|
265 |
Except states 2b and 5, which explicitly state that they are |
266 |
*pending synchronization*, all other states are considered *synchronized* |
267 |
**Synchronization** refers to all external communication |
268 |
(i.e. not within the limits to) required |
269 |
|
270 |
|
271 |
|
272 |
Project |
273 |
------- |
274 |
A **project** is created or modified upon the approval of an application. |
275 |
It is initialized with the *definition* contained in the application, |
276 |
and grants resources as specified, and acquires all the extra state needed. |
277 |
The *definition* of a project does not belong to its mutable state, |
278 |
and can only be changed by approving a new application. |
279 |
The attributes for a project are: |
280 |
|
281 |
``serial`` |
282 |
*a unique serial number identifying the project* |
283 |
|
284 |
``application`` |
285 |
*the last application that was successfully synchronized with Quotaholder.* |
286 |
|
287 |
``creation_date`` |
288 |
*when the project was created (i.e. was first approved)* |
289 |
|
290 |
``last_approval_date`` |
291 |
*when was the last approval (i.e. creation or modification). |
292 |
Null if the project has not been approved or has been suspended.* |
293 |
|
294 |
``deactivation_start_date`` |
295 |
*when the project was ordered to deactivate, |
296 |
and declared out of sync for its resource grants to be removed. |
297 |
(is null if the project has not been ordered to deactivate)* |
298 |
|
299 |
``deactivation_date`` |
300 |
*when the project deactivation was actually registered as completed |
301 |
by the service following the successful revocation of resource grants. |
302 |
(is null if the project has not been deactivated)* |
303 |
|
304 |
``deactivation_reason`` |
305 |
*Text indicating indicating the reason for deactivation.* |
306 |
|
307 |
``members`` |
308 |
*the set of memberships for this project* |
309 |
|
310 |
|
311 |
Rules |
312 |
----- |
313 |
|
314 |
1. **Valid projects** |
315 |
|
316 |
Projects are valid only if they violate none of the rules |
317 |
|
318 |
2. **Active projects** |
319 |
|
320 |
A project is declared **active** when its resource grants and |
321 |
general policy is in effect (even if partially), |
322 |
and no order of deactivation has been given. |
323 |
|
324 |
A valid project can be active if and only if |
325 |
- its ``deactivation_start_date`` is null |
326 |
|
327 |
2. **Inactive projects** |
328 |
|
329 |
A valid project is declared **inactive** when its resource grants |
330 |
and general policy is not in effect, or is in effect and |
331 |
|
332 |
A valid project is inactive if and only if, |
333 |
its ``deactivation_start_date`` is not null |
334 |
|
335 |
3. **Project states** |
336 |
|
337 |
The states of a project that are significant from a control flow aspect, |
338 |
are the following: |
339 |
|
340 |
:(0): pending approval |
341 |
:(1a): active, pending definition sync |
342 |
:(1b): active, pending membership sync |
343 |
:(1c): active, pending total sync |
344 |
:(2): active |
345 |
:(3a): inactive, pending definition sync |
346 |
:(3b): inactive, pending membership sync |
347 |
:(3c): inactive, pending total sync |
348 |
:(4): inactive |
349 |
|
350 |
|
351 |
4. **Synchronization status** |
352 |
|
353 |
The status of the project's synchronization with Quotaholder |
354 |
and other remote services, |
355 |
can be either **synchronized** or **unsyncrhonized**. |
356 |
|
357 |
Semantically, the project becomes synchronized when its application |
358 |
definition has been fully implemented and committed to quotaholder, |
359 |
and all its memberships are also synchronized. |
360 |
|
361 |
The active project loses its synchronization on two occasions. |
362 |
On the approval of a new application modifying the project, |
363 |
and on the addition or removal of any of its memberships. |
364 |
|
365 |
In general, also considering projects being deactivated, |
366 |
a project is declared synchronized if and only if: |
367 |
|
368 |
- None of its Memberships is unsynchronized |
369 |
- ``deactivation_start_date`` is null or ``deactivation_date`` is set |
370 |
|
371 |
5. **Unique project names** |
372 |
|
373 |
The project name (as specified in its application's definition) |
374 |
must be unique among all *alive* projects. |
375 |
|
376 |
6. **Inconsistent dates** |
377 |
|
378 |
If either ``creation_date`` or ``last_approval_date`` |
379 |
is in the future, the project state is declared **inconsistent** |
380 |
but the project is still considered created or approved, respectively. |
381 |
|
382 |
If ``deactivation_date`` is in the future, the project is declared |
383 |
**inconsistent** but the project is still considered inactive. |
384 |
|
385 |
7. **No project without application** |
386 |
|
387 |
A project can only exist in reference of the application that has defined it. |
388 |
The attributes in the definition such as its name, resource grants |
389 |
cannot be modified without a new application that must be approved. |
390 |
|
391 |
8. **Creating and modifying projects with follow-up applications** |
392 |
|
393 |
Every application for a project can be followed up with another one. |
394 |
The new application points back to it with its ``precursor`` attribute. |
395 |
|
396 |
This means that an applicant can update their application |
397 |
before it has been approved. |
398 |
|
399 |
Apart from an applicant, whoever can approve the project, |
400 |
can also post a follow up application for it, modifying |
401 |
some aspects of the definition and approve that instead. |
402 |
For example, a user might request for 100 GB storage, |
403 |
but the Service may approve a project with only 80GB, |
404 |
via a follow up application. |
405 |
|
406 |
If the precursor of an application is not associated with a project |
407 |
(i.e. no project references it as its defining ``last_application_approved``), |
408 |
then a new project entry is created and initialized. |
409 |
|
410 |
If the precursor of an application *is* associated with a valid project, |
411 |
then the same project entry is used and is re-initialized according |
412 |
to the new application's definition. |
413 |
The project is made active (if inactive) and its previous state |
414 |
is preserved (mainly, the member set). |
415 |
|
416 |
|
417 |
Scenarios |
418 |
--------- |
419 |
|
420 |
Project applicant not confident for his numbers |
421 |
''''''''''''''''''''''''''''''''''''''''''''''' |
422 |
Researcher requests resources to create a cluster for a protein-folding |
423 |
computation experiment. He knows how exactly how many machines, |
424 |
memory, and disk he needs but is not certain how many CPU cores |
425 |
he should request. |
426 |
|
427 |
He leaves the corresponding resource unspecified, |
428 |
and leaves a comment noting the issue. |
429 |
|
430 |
The project administrator responsible for the application |
431 |
uses his expertise, and/or consults others to formulate an appropriate |
432 |
limit for the resource. |
433 |
Then he modifies the (not-yet-approved) project, fills in the resource limit, |
434 |
then submits and approves the new application. |
435 |
|
436 |
|
437 |
Using participation to support an project application |
438 |
''''''''''''''''''''''''''''''''''''''''''''''''''''' |
439 |
An applicant knows that his application will be rejected unless |
440 |
a lot of people support it. |
441 |
|
442 |
Therefore, he applies for a project with no resource grants, |
443 |
and notes his rationale in the comments. |
444 |
|
445 |
The project administrator accepts his application in good faith, |
446 |
and then the project begins to accept members. |
447 |
|
448 |
Once many and important enough members have joined, |
449 |
the project leader modifies the project with a new application |
450 |
that now includes significant resource grants. |
451 |
|
452 |
The project administrator reviews the application and membership list, |
453 |
and is convinced that the project deserves the grants. |
454 |
However, he wants to make sure that this remains so, |
455 |
by ensuring that the membership of the project cannot |
456 |
include other users without further review. |
457 |
|
458 |
Therefore, he further modifies the (not-yet-approved) application, |
459 |
and sets the member accept policy to be 'closed', that is, |
460 |
that no new members may join the project. |
461 |
Then he sumbits and approves the application. |
462 |
|
463 |
|
464 |
|
465 |
-------- gtsouk REVIEW STOPS HERE --------- |
466 |
|
467 |
Procedures |
468 |
---------- |
469 |
|
470 |
The following procedures are considered essential primitives |
471 |
for the function of the Projects feature, and should constitute |
472 |
a natural *internal* API to it. This is not a public API for users, |
473 |
although the public one should be thinly layered on top of it. |
474 |
|
475 |
For quotaholder interactions, a primitive to synchronize quotaholder |
476 |
with a user's resource quotas is assumed to be available. |
477 |
|
478 |
#. Submit an application |
479 |
|
480 |
Every user might submit an application to create or modify a project. |
481 |
There are no inherent constraints. |
482 |
However, policy may impose restrictions. |
483 |
|
484 |
#. Retrieve applications |
485 |
|
486 |
Applications should be retrievable by serial, by applicant, |
487 |
and by approval status. |
488 |
Because applications are immutable, some kind of extra state |
489 |
is needed in order to index by approval status. |
490 |
For example, one might instantiate a mutable object for approval state, |
491 |
with attributes (``approval_serial``, ``approval_state``, ``approval_data``) |
492 |
|
493 |
#. Approve an application (create or modify a Project) |
494 |
|
495 |
Application approval must atomically perform and commit all of the following: |
496 |
- create or modify the project at its specified definition |
497 |
- set the project state to be pending synchronization with quotaholder |
498 |
|
499 |
After committing, the procedure to synchronize the project (see below) |
500 |
should be called. |
501 |
|
502 |
#. Retrieve projects |
503 |
|
504 |
Projects should be retrievable by serial, by owner, |
505 |
by application, by life status, and by synchronization status. |
506 |
|
507 |
*By application* means that given an application, one must be able |
508 |
to retrieve the project that has it approved, if any. |
509 |
|
510 |
#. Add or remove a user to a project |
511 |
|
512 |
When a new membership is created for a project or |
513 |
an existing one is modified, then in the same |
514 |
transaction the project must be declared *unsynchronized*. |
515 |
|
516 |
#. Suspend a project |
517 |
|
518 |
A project is suspended by setting ``last_approval_date`` to None. |
519 |
|
520 |
#. Terminate a project |
521 |
|
522 |
Project termination must atomically perform and commit all of the following: |
523 |
- set the project ``termination_start_date`` |
524 |
- set the ``termination_date`` to null (this is a redundant safeguard) |
525 |
in order to mark the project as pending termination synchronization. |
526 |
|
527 |
After committing, the procedure to synchronize the project (see below) |
528 |
should be called. |
529 |
|
530 |
#. Synchronize a membership with quotaholder |
531 |
|
532 |
This procedure is not required to be an independent procedure |
533 |
available for individual memberships. |
534 |
That is, if user synchronization takes care of all memberships, |
535 |
one can call user synchronization instead of membership synchronization. |
536 |
|
537 |
#. Synchronize a user with quotaholder |
538 |
|
539 |
User synchronization is equivalent to the synchronization of |
540 |
all the user's memberships. |
541 |
|
542 |
#. Synchronize a project with quotaholder |
543 |
|
544 |
Project synchronization is equivelent to the synchronization of |
545 |
either all its memberships, or all its members. |
546 |
|
547 |
|
548 |
|
549 |
View elements |
550 |
------------- |
551 |
In this context, view elements refer to (at best effort) self-contained |
552 |
elements that display information for and offer interaction with their |
553 |
underlying objects. A final view, such as a web page, or console command |
554 |
output, may include several elements. |
555 |
|
556 |
Interaction with the feature is going through several view elements. |
557 |
Each element has two pieces of input, |
558 |
the *objects being referenced*, and the *accessing user*, |
559 |
and its appearance and function is accordingly parameterized. |
560 |
|
561 |
Each *project element* may need to display information |
562 |
that internally belongs to its linked object, |
563 |
such as its *definition* or *memberships*. |
564 |
|
565 |
A proposed list of elements is following. |
566 |
The elements are only considered building blocks |
567 |
for views and not full views by themselves. |
568 |
More on views later on. |
569 |
|
570 |
#. Project list element |
571 |
|
572 |
This represents a list of *projects.* |
573 |
Technically, most of information about a project resides in its *application*. |
574 |
|
575 |
In the case of a *pending application*, |
576 |
one that has not been approved yet, |
577 |
then the row is filled with data from the |
578 |
current (approved and effective) application, |
579 |
or they are left blank if no such project exists yet. |
580 |
|
581 |
Additionally, the column that displays the pending status |
582 |
of the project (creation or modification), should be a link |
583 |
to the corresponding application detail element. |
584 |
|
585 |
#. Membership element |
586 |
|
587 |
A list of project members for a single (or maybe multiple) projects. |
588 |
The list must not display user emails if the viewer is not the owner |
589 |
of the group, or a service administrator. |
590 |
|
591 |
#. Application details element |
592 |
|
593 |
Displays all application details in full, |
594 |
with a link to the (alive) project it defines, if any, |
595 |
or a pending notice, |
596 |
or an obsolescence note if the project it defined was since modified, |
597 |
or a rejection notice. |
598 |
|
599 |
#. Project details element |
600 |
|
601 |
This element contains both details for the defining application |
602 |
and for the current state of the project. |
603 |
|
604 |
Details for the current state of the project may include |
605 |
statistics on membership and resource usage. |
606 |
|
607 |
#. Application form element |
608 |
|
609 |
It submits a new application. |
610 |
If the application is in reference |
611 |
to previous application (the ``precursor``) |
612 |
then the form should be initialized to be a copy of it. |
613 |
|
614 |
Otherwise, it may be blank, or initialized with defaults from a template. |
615 |
|
616 |
#. Project search element |
617 |
|
618 |
This is an entry point to a listing, with configurable filtering. |
619 |
|
620 |
#. History and statistics view |
621 |
|
622 |
No specification currently. |
623 |
|