Statistics
| Branch: | Tag: | Revision:

root / snf-astakos-app / docs / projects_specs.rst @ d41f9720

History | View | Annotate | Download (21.8 kB)

1
Synnefo Projects: General Design Notes
2
======================================
3

    
4
Projects can be thought of as "contracts" between the infrastructure
5
and the members of the Project, so that members can allocate and use
6
resources for a specified period in time.
7
For every Project there is an *application*, and a *membership set*.
8
The application must be approved before the Project "contract"
9
comes in effect, and before any members join.
10

    
11
Applications contain a Project *definition* that formally includes
12
all the policy of the Project "contract", such as name, ownership,
13
dates, and resource limits.
14
Applications and the definitions they contain are, like contracts,
15
immutable objects that can be tracked and effected in their entirety.
16
Every change made to a pending application or an existing Project
17
must be applied for through a new application.
18

    
19
Project Synchronization
20
-----------------------
21
A Project has two kinds of effects, global and per-membership.
22
Global effects are encoded within the Project object itself,
23
and may include services (e.g. dns name, website, forum/collaboration),
24
project-wide resources (e.g. VMs, IPs, diskspace), etc.
25
Per-membership effects are encoded within Membership objects,
26
and are prescribed by the policy in the project definition.
27

    
28
By design, the effects of the project are expected to extend
29
beyond the database system that records the project approval,
30
therefore these effects cannot be relied to be implemented
31
instantly and atomically upon request.
32
For example, project creation or modification may involve
33
updating resource management services, notification services,
34
project website services, which may all have their own separate
35
database systems.
36

    
37
Consequently, once a policy is (atomically) registered as being in effect,
38
it must be propagated for implementation with separate (non-atomic) procedures,
39
collectively called Project Synchronization.
40

    
41
Currently, the general approach for Synchronization of synnefo Projects,
42
is to maintain appropriate states for the objects encoding policy effects
43
(i.e. Project and Membership objects), and execute careful transitions
44
among those states, respecting semanting dependencies and limitations,
45
and remote system accesses.
46
For example, a project cannot be considered "synchronized" unless
47
all its Memberships are also considered "synchronized",
48
or you cannot declare a Membership as synchronized if the newly-set
49
quotas have not been acknowledged by the remote quota service.
50

    
51
Creating and Modifying Projects
52
-------------------------------
53
Projects are created and modified by issuing a corresponding application.
54
Applications that refer to the same Project are chained by including
55
a reference to the precursor application into the new one.
56
Applications that do not specify a precursor always create a new Project
57
(that is, a new project object with a new membership set).
58
Project applications that have not yet been approved may also be modified.
59
When an unapproved application is modified (ie. it is succeeded by another),
60
it may or may not be automatically rejected, depending on policy.
61

    
62
Projects once created, are expected to always remain in record,
63
even if they have been deactivated (i.e. their policies removed from effect).
64
Deactivating a project may be the result of a policy-specific action,
65
such as *termination* on a pre-defined date or *suspension* following
66
an administrative decision.
67
Deactivated projects retain both their Definition and their Membership sets,
68
allowing them to be reactivated by another kind of policy-specific action.
69

    
70
Adding and Removing Project Members
71
-----------------------------------
72
Memberships once created, are also expected to remain in historical record,
73
even if they have been marked as removed by a policy-specific action,
74
such as the user leaving the project,
75
or the project owner suspending, or terminating them altogether.
76

    
77

    
78

    
79
Reference Schema for Projects
80
=============================
81

    
82
General Terms
83
-------------
84
For clarity and precision, we (try to) use different terms for similar meanings.
85
We also separate low-level technical primitives from related higher-level
86
policy actions that include them.
87

    
88
synchronize, synchronized, synchronization
89
    A low-level term for Projects and Memberships.
90
    Refers to the controlled (non-atomic) process that implements
91
    a newly modified policy (i.e. adds or removes policy effects
92
    such as quota limits).
93

    
94
activate, deactivate, active, inactive
95
    A low-level term for Projects and Memberships.
96
    A Project or Membership is considered active as long as
97
    its policy is in effect.
98

    
99
suspend, suspended, suspension
100
    A policy term for Projects and Memberships,
101
    implying temporary deactivation by an administrative action
102
    (e.g. abuse report, limits violation).
103

    
104
terminate, terminated, termination
105
    A policyu term for Projects and Memberships,
106
    implying permanent deactivation, especially
107
    according to a pre-defined end-of-life event
108
    (e.g. project/contract "expiration").
109

    
110
add or remove membership
111
    Low-level terms for managing memberships to projects.
112

    
113
join or leave project
114
    Policy terms for when users request their addition or removal
115
    from a project membership set.
116

    
117
accept or reject membership request
118
    Policy terms for a project administrator to decide on
119
    join or leave requests.
120

    
121
approve or disapprove project
122
    Policy terms for service administrators to decide whether
123
    to create/activate a project according or not
124

    
125
alter project
126
    Low-level term for changes made to the global (i.e. not Membership)
127
    project status (e.g. new application, suspension).
128

    
129
modify project
130
    Policy term for submitting a new application as a successor
131
    to an existing one, to alter the definition of a project.
132

    
133
project leader
134
    The user who has authority over a project,
135
    to accept or reject membership requests,
136
    or to perform other actions according to policy.
137

    
138
project applicant
139
    The user who submits a project application for creation
140
    or modification of a project.
141
    The applicant can be a different user than the leader.
142

    
143
project administrator
144
    A user who has authority to approve, disapprove, and modify
145
    projects of a certain class (e.g. according to their domain names).
146

    
147

    
148
Definition
149
----------
150
A **definition** for a project, which represents the initialization
151
or modification of a project, has the following attributes:
152

    
153
``name``
154
    *in dot-hierarchical dns-like form*
155

    
156
``description``
157
    *text describing the project for the public*
158

    
159
``start_date``
160
    *when the project is requested to become active*
161

    
162
``end_date``
163
    *when the project is to be deactivatedended*
164

    
165
``member_join_policy``
166
    *an enumeration of policies on how new join requests are to be accepted.
167
    The policies include the following:*
168

    
169
    ``auto_accept``
170
        *new join requests are automatically accepted by the system*
171

    
172
    ``owner_accepts``
173
        *new join requests must be accepted by the owner of the project*
174

    
175
    ``closed``
176
        *no new members can join the project, even if old ones leave*
177

    
178
``member_leave_policy``
179
    *an enumeration of policies on how new leave requests are to be accepted.
180
    The policies include the following:*
181

    
182
    ``auto_accept``
183
        *leave requests are automatically accepted by the system*
184

    
185
    ``owner_accepts``
186
        *leave requests must be accepted by the owner of the project**
187

    
188
    ``closed``
189
        *no member can leave the project*
190

    
191
``limit_on_member_count``
192
    *the maximum number of members that can be admitted to the project*
193

    
194
``limits_on_resources``
195
    *a set of limits on the total resources the group can grant to its members*
196

    
197
``resource_grants``
198
    *a set of resource usage limits each member is granted by the project*
199

    
200

    
201

    
202
Application for a Project
203
-------------------------
204
An **application** for a project must be issued by a user and
205
*approved* by the service before any resources are granted.
206
Its attributes are:
207

    
208
``serial``
209
    *a unique serial number identifying the application*
210

    
211
``applicant``
212
    *who applies for creation or modification of a project*
213

    
214
``owner``
215
    *the requested owner of the project*
216

    
217
``comments``
218
    *text comments for the application*
219

    
220
``definition``
221
    *the definition of the project being requested*
222

    
223
``issue_date``
224
    *when the application was submitted*
225

    
226
``precursor_application``
227
    *a reference by serial to a previous application which this application
228
    requests to modify. It can be null if there is no precursor.*
229

    
230
An application is immutable in its attributes above,
231
but its status depends on how it has been processed.
232
The *application status* can be:
233

    
234
    :(1):   pending
235
    :(2a):  approved
236
    :(2b):  rejected
237
    :(3):   replaced
238

    
239
When an application becomes *approved* and therefore defines
240
a the project, its precursor (if any) must atomically be set to *replaced*.
241

    
242

    
243

    
244
Project Membership
245
-------------------------
246
A *project membership* maps a user to a project and holds state for this mapping.
247
There are no inherent constraints to this mapping,
248
any user might be a member to any project.
249

    
250
The **state** of membership can be:
251

    
252
    :(1):   *pending acceptance*
253
    :(2a):  *rejected*
254
    :(2b):  *accepted, pending synchronization*
255
    :(3):   *active*
256
    :(4):   *pending removal*
257
    :(5):   *removed, pending synchronization*
258
    :(6):   *removed*
259

    
260
The transitions from 2b to 3, and 5 to 6, must first commit
261
their starting state and then only update to the next state
262
after the *Synchronize Membership* procedure has been
263
acknowledged as successful.
264

    
265
Except states 2b and 5, which explicitly state that they are
266
*pending synchronization*, all other states are considered *synchronized*
267
**Synchronization** refers to all external communication 
268
(i.e. not within the limits to) required
269

    
270

    
271

    
272
Project
273
-------
274
A **project** is created or modified upon the approval of an application.
275
It is initialized with the *definition* contained in the application,
276
and grants resources as specified, and acquires all the extra state needed.
277
The *definition* of a project does not belong to its mutable state,
278
and can only be changed by approving a new application.
279
The attributes for a project are:
280

    
281
``serial``
282
    *a unique serial number identifying the project*
283

    
284
``application``
285
    *the last application that was successfully synchronized with Quotaholder.*
286

    
287
``creation_date``
288
    *when the project was created (i.e. was first approved)*
289

    
290
``last_approval_date``
291
    *when was the last approval (i.e. creation or modification).
292
    Null if the project has not been approved or has been suspended.*
293

    
294
``deactivation_start_date``
295
    *when the project was ordered to deactivate,
296
    and declared out of sync for its resource grants to be removed.
297
    (is null if the project has not been ordered to deactivate)*
298

    
299
``deactivation_date``
300
    *when the project deactivation was actually registered as completed
301
    by the service following the successful revocation of resource grants.
302
    (is null if the project has not been deactivated)*
303

    
304
``deactivation_reason``
305
    *Text indicating indicating the reason for deactivation.*
306

    
307
``members``
308
    *the set of memberships for this project*
309

    
310

    
311
Rules
312
-----
313

    
314
1. **Valid projects**
315

    
316
    Projects are valid only if they violate none of the rules
317

    
318
2. **Active projects**
319

    
320
    A project is declared **active** when its resource grants and
321
    general policy is in effect (even if partially),
322
    and no order of deactivation has been given.
323

    
324
    A valid project can be active if and only if
325
    - its ``deactivation_start_date`` is null
326

    
327
2. **Inactive projects**
328

    
329
    A valid project is declared **inactive** when its resource grants
330
    and general policy is not in effect, or is in effect and
331

    
332
    A valid project is inactive if and only if,
333
    its ``deactivation_start_date`` is not null
334

    
335
3. **Project states**
336

    
337
    The states of a project that are significant from a control flow aspect,
338
    are the following:
339

    
340
    :(0):   pending approval
341
    :(1a):  active, pending definition sync
342
    :(1b):  active, pending membership sync
343
    :(1c):  active, pending total sync
344
    :(2):   active
345
    :(3a):  inactive, pending definition sync
346
    :(3b):  inactive, pending membership sync
347
    :(3c):  inactive, pending total sync
348
    :(4):   inactive
349

    
350

    
351
4. **Synchronization status**
352

    
353
    The status of the project's synchronization with Quotaholder
354
    and other remote services,
355
    can be either **synchronized** or **unsyncrhonized**.
356

    
357
    Semantically, the project becomes synchronized when its application
358
    definition has been fully implemented and committed to quotaholder,
359
    and all its memberships are also synchronized.
360

    
361
    The active project loses its synchronization on two occasions.
362
    On the approval of a new application modifying the project,
363
    and on the addition or removal of any of its memberships.
364

    
365
    In general, also considering projects being deactivated,
366
    a project is declared synchronized if and only if:
367

    
368
    - None of its Memberships is unsynchronized
369
    - ``deactivation_start_date`` is null or ``deactivation_date`` is set
370

    
371
5. **Unique project names**
372

    
373
    The project name (as specified in its application's definition)
374
    must be unique among all *alive* projects.
375

    
376
6. **Inconsistent dates**
377

    
378
    If either ``creation_date`` or ``last_approval_date``
379
    is in the future, the project state is declared **inconsistent**
380
    but the project is still considered created or approved, respectively.
381

    
382
    If ``deactivation_date`` is in the future, the project is declared
383
    **inconsistent** but the project is still considered inactive.
384

    
385
7. **No project without application**
386

    
387
    A project can only exist in reference of the application that has defined it.
388
    The attributes in the definition such as its name, resource grants
389
    cannot be modified without a new application that must be approved.
390

    
391
8. **Creating and modifying projects with follow-up applications**
392

    
393
    Every application for a project can be followed up with another one.
394
    The new application points back to it with its ``precursor`` attribute.
395

    
396
    This means that an applicant can update their application
397
    before it has been approved.
398

    
399
    Apart from an applicant, whoever can approve the project,
400
    can also post a follow up application for it, modifying
401
    some aspects of the definition and approve that instead.
402
    For example, a user might request for 100 GB storage,
403
    but the Service may approve a project with only 80GB,
404
    via a follow up application.
405

    
406
    If the precursor of an application is not associated with a project
407
    (i.e. no project references it as its defining ``last_application_approved``),
408
    then a new project entry is created and initialized.
409

    
410
    If the precursor of an application *is* associated with a valid project,
411
    then the same project entry is used and is re-initialized according
412
    to the new application's definition.
413
    The project is made active (if inactive) and its previous state
414
    is preserved (mainly, the member set).
415

    
416

    
417
Scenarios
418
---------
419

    
420
Project applicant not confident for his numbers
421
'''''''''''''''''''''''''''''''''''''''''''''''
422
Researcher requests resources to create a cluster for a protein-folding
423
computation experiment. He knows how exactly how many machines,
424
memory, and disk he needs but is not certain how many CPU cores
425
he should request.
426

    
427
He leaves the corresponding resource unspecified,
428
and leaves a comment noting the issue.
429

    
430
The project administrator responsible for the application
431
uses his expertise, and/or consults others to formulate an appropriate
432
limit for the resource.
433
Then he modifies the (not-yet-approved) project, fills in the resource limit,
434
then submits and approves the new application.
435

    
436

    
437
Using participation to support an project application
438
'''''''''''''''''''''''''''''''''''''''''''''''''''''
439
An applicant knows that his application will be rejected unless
440
a lot of people support it.
441

    
442
Therefore, he applies for a project with no resource grants,
443
and notes his rationale in the comments.
444

    
445
The project administrator accepts his application in good faith,
446
and then the project begins to accept members.
447

    
448
Once many and important enough members have joined,
449
the project leader modifies the project with a new application
450
that now includes significant resource grants.
451

    
452
The project administrator reviews the application and membership list,
453
and is convinced that the project deserves the grants.
454
However, he wants to make sure that this remains so,
455
by ensuring that the membership of the project cannot
456
include other users without further review.
457

    
458
Therefore, he further modifies the (not-yet-approved) application,
459
and sets the member accept policy to be 'closed', that is,
460
that no new members may join the project.
461
Then he sumbits and approves the application.
462

    
463

    
464

    
465
-------- gtsouk REVIEW STOPS HERE ---------
466

    
467
Procedures
468
----------
469

    
470
The following procedures are considered essential primitives
471
for the function of the Projects feature, and should constitute
472
a natural *internal* API to it. This is not a public API for users,
473
although the public one should be thinly layered on top of it.
474

    
475
For quotaholder interactions, a primitive to synchronize quotaholder
476
with a user's resource quotas is assumed to be available.
477

    
478
#. Submit an application
479

    
480
   Every user might submit an application to create or modify a project.
481
   There are no inherent constraints.
482
   However, policy may impose restrictions.
483

    
484
#. Retrieve applications
485

    
486
   Applications should be retrievable by serial, by applicant,
487
   and by approval status.
488
   Because applications are immutable, some kind of extra state
489
   is needed in order to index by approval status.
490
   For example, one might instantiate a mutable object for approval state,
491
   with attributes (``approval_serial``, ``approval_state``, ``approval_data``)
492

    
493
#. Approve an application (create or modify a Project)
494

    
495
   Application approval must atomically perform and commit all of the following:
496
   - create or modify the project at its specified definition
497
   - set the project state to be pending synchronization with quotaholder
498

    
499
   After committing, the procedure to synchronize the project (see below)
500
   should be called.
501

    
502
#. Retrieve projects
503

    
504
   Projects should be retrievable by serial, by owner,
505
   by application, by life status, and by synchronization status.
506

    
507
   *By application* means that given an application, one must be able
508
   to retrieve the project that has it approved, if any.
509

    
510
#. Add or remove a user to a project
511

    
512
   When a new membership is created for a project or
513
   an existing one is modified, then in the same
514
   transaction the project must be declared *unsynchronized*.
515

    
516
#. Suspend a project
517

    
518
   A project is suspended by setting ``last_approval_date`` to None.
519

    
520
#. Terminate a project
521

    
522
   Project termination must atomically perform and commit all of the following:
523
   - set the project ``termination_start_date``
524
   - set the ``termination_date`` to null (this is a redundant safeguard)
525
   in order to mark the project as pending termination synchronization.
526

    
527
   After committing, the procedure to synchronize the project (see below)
528
   should be called.
529

    
530
#. Synchronize a membership with quotaholder
531

    
532
   This procedure is not required to be an independent procedure
533
   available for individual memberships.
534
   That is, if user synchronization takes care of all memberships,
535
   one can call user synchronization instead of membership synchronization.
536

    
537
#. Synchronize a user with quotaholder
538

    
539
   User synchronization is equivalent to the synchronization of
540
   all the user's memberships.
541

    
542
#. Synchronize a project with quotaholder
543

    
544
   Project synchronization is equivelent to the synchronization of
545
   either all its memberships, or all its members.
546

    
547

    
548

    
549
View elements
550
-------------
551
In this context, view elements refer to (at best effort) self-contained
552
elements that display information for and offer interaction with their
553
underlying objects. A final view, such as a web page, or console command
554
output, may include several elements.
555

    
556
Interaction with the feature is going through several view elements.
557
Each element has two pieces of input,
558
the *objects being referenced*, and the *accessing user*,
559
and its appearance and function is accordingly parameterized.
560

    
561
Each *project element* may need to display information
562
that internally belongs to its linked object,
563
such as its *definition* or *memberships*.
564

    
565
A proposed list of elements is following.
566
The elements are only considered building blocks
567
for views and not full views by themselves.
568
More on views later on.
569

    
570
#. Project list element
571

    
572
   This represents a list of *projects.*
573
   Technically, most of information about a project resides in its *application*.
574

    
575
   In the case of a *pending application*,
576
   one that has not been approved yet,
577
   then the row is filled with data from the
578
   current (approved and effective) application,
579
   or they are left blank if no such project exists yet.
580

    
581
   Additionally, the column that displays the pending status
582
   of the project (creation or modification), should be a link
583
   to the corresponding application detail element.
584

    
585
#. Membership element
586

    
587
   A list of project members for a single (or maybe multiple) projects.
588
   The list must not display user emails if the viewer is not the owner
589
   of the group, or a service administrator.
590

    
591
#. Application details element
592

    
593
   Displays all application details in full,
594
   with a link to the (alive) project it defines, if any,
595
   or a pending notice,
596
   or an obsolescence note if the project it defined was since modified,
597
   or a rejection notice.
598

    
599
#. Project details element
600

    
601
   This element contains both details for the defining application
602
   and for the current state of the project.
603

    
604
   Details for the current state of the project may include
605
   statistics on membership and resource usage.
606

    
607
#. Application form element
608

    
609
   It submits a new application.
610
   If the application is in reference
611
   to previous application (the ``precursor``)
612
   then the form should be initialized to be a copy of it.
613

    
614
   Otherwise, it may be blank, or initialized with defaults from a template.
615

    
616
#. Project search element
617

    
618
   This is an entry point to a listing, with configurable filtering.
619

    
620
#. History and statistics view
621

    
622
   No specification currently.
623