root / docs / dev-guide.rst @ 114b028e
History | View | Annotate | Download (11.1 kB)
1 | bc055d09 | Constantinos Venetsanopoulos | .. _dev-guide: |
---|---|---|---|
2 | bc055d09 | Constantinos Venetsanopoulos | |
3 | bc055d09 | Constantinos Venetsanopoulos | Synnefo Developer's Guide |
4 | bc055d09 | Constantinos Venetsanopoulos | ^^^^^^^^^^^^^^^^^^^^^^^^^ |
5 | bc055d09 | Constantinos Venetsanopoulos | |
6 | 114b028e | marioskogias | This is the complete Synnefo Developer's Guide. |
7 | 114b028e | marioskogias | |
8 | 114b028e | marioskogias | Environment set up |
9 | 114b028e | marioskogias | ================== |
10 | 114b028e | marioskogias | |
11 | 114b028e | marioskogias | First of all you have to set up a developing environment for Synnefo. |
12 | 114b028e | marioskogias | |
13 | 114b028e | marioskogias | **1. Create a new Squeeze VM** |
14 | 114b028e | marioskogias | |
15 | 114b028e | marioskogias | **2. Build your own Synnefo installation** |
16 | 114b028e | marioskogias | |
17 | 114b028e | marioskogias | Follow the instructions `here <http://www.synnefo.org/docs/synnefo/latest/quick-install-guide.html>`_ |
18 | 114b028e | marioskogias | to build Synnefo on a single node using ``snf-deploy``. |
19 | 114b028e | marioskogias | |
20 | 114b028e | marioskogias | **3. Install GitPython** |
21 | 114b028e | marioskogias | |
22 | 114b028e | marioskogias | .. code-block:: console |
23 | 114b028e | marioskogias | |
24 | 114b028e | marioskogias | # pip install gitpython |
25 | 114b028e | marioskogias | |
26 | 114b028e | marioskogias | **4. Install devflow** |
27 | 114b028e | marioskogias | |
28 | 114b028e | marioskogias | Devflow is a tool to manage versions, helps implement the git flow development process, |
29 | 114b028e | marioskogias | and builds Python and Debian packages. You will need it to create your code's version. |
30 | 114b028e | marioskogias | |
31 | 114b028e | marioskogias | .. code-block:: console |
32 | 114b028e | marioskogias | |
33 | 114b028e | marioskogias | # pip install devflow |
34 | 114b028e | marioskogias | |
35 | 114b028e | marioskogias | **5. Get Synnefo code** |
36 | 114b028e | marioskogias | |
37 | 114b028e | marioskogias | First you need to install git |
38 | 114b028e | marioskogias | |
39 | 114b028e | marioskogias | .. code-block:: console |
40 | 114b028e | marioskogias | |
41 | 114b028e | marioskogias | # apt-get install git |
42 | 114b028e | marioskogias | |
43 | 114b028e | marioskogias | And now get the Synnefo code from the official Synnefo repository |
44 | 114b028e | marioskogias | |
45 | 114b028e | marioskogias | .. code-block:: console |
46 | 114b028e | marioskogias | |
47 | 114b028e | marioskogias | $ git clone https://code.grnet.gr/git/synnefo |
48 | 114b028e | marioskogias | |
49 | 114b028e | marioskogias | Make sure you did the previous as a regular user. Otherwise you will have problems |
50 | 114b028e | marioskogias | with file permissions when deploying. |
51 | 114b028e | marioskogias | |
52 | 114b028e | marioskogias | **6. Code and deploy** |
53 | 114b028e | marioskogias | |
54 | 114b028e | marioskogias | 1. Configure the version |
55 | 114b028e | marioskogias | |
56 | 114b028e | marioskogias | .. code-block:: console |
57 | 114b028e | marioskogias | |
58 | 114b028e | marioskogias | $ devflow-update-version |
59 | 114b028e | marioskogias | |
60 | 114b028e | marioskogias | 2. Code |
61 | 114b028e | marioskogias | 3. In every component you change run |
62 | 114b028e | marioskogias | |
63 | 114b028e | marioskogias | .. code-block:: console |
64 | 114b028e | marioskogias | |
65 | 114b028e | marioskogias | # python setup.py develop |
66 | 114b028e | marioskogias | |
67 | 114b028e | marioskogias | 4. Refresh the web page and see your changes |
68 | 114b028e | marioskogias | |
69 | 114b028e | marioskogias | Synnefo REST APIs |
70 | 114b028e | marioskogias | ================= |
71 | 114b028e | marioskogias | |
72 | 114b028e | marioskogias | Here, we document all Synnefo |
73 | 991d0507 | Constantinos Venetsanopoulos | REST APIs, to allow external developers write independent tools that interact |
74 | 991d0507 | Constantinos Venetsanopoulos | with Synnefo. |
75 | bc055d09 | Constantinos Venetsanopoulos | |
76 | 991d0507 | Constantinos Venetsanopoulos | Synnefo exposes the OpenStack APIs for all its operations. Also, extensions |
77 | 991d0507 | Constantinos Venetsanopoulos | have been written for advanced operations wherever needed, and minor changes |
78 | 991d0507 | Constantinos Venetsanopoulos | for things that were missing or change frequently. |
79 | 8f9976c6 | Constantinos Venetsanopoulos | |
80 | 73ebcd68 | Constantinos Venetsanopoulos | Most Synnefo services have a corresponding OpenStack API: |
81 | 991d0507 | Constantinos Venetsanopoulos | |
82 | 991d0507 | Constantinos Venetsanopoulos | | Cyclades/Compute Service -> OpenStack Compute API |
83 | 991d0507 | Constantinos Venetsanopoulos | | Cyclades/Network Service -> OpenStack Compute/Network API (not Quantum yet) |
84 | 73ebcd68 | Constantinos Venetsanopoulos | | Cyclades/Image Service -> OpenStack Glance API |
85 | 991d0507 | Constantinos Venetsanopoulos | | Pithos/Storage Service -> OpenStack Object Store API |
86 | 991d0507 | Constantinos Venetsanopoulos | | Astakos/Identity Service -> Proprietary, moving to OpenStack Keystone API |
87 | 73ebcd68 | Constantinos Venetsanopoulos | | Astakos/Quota Service -> Proprietary API |
88 | 73ebcd68 | Constantinos Venetsanopoulos | | Astakos/Resource Service -> Proprietary API |
89 | 991d0507 | Constantinos Venetsanopoulos | |
90 | 991d0507 | Constantinos Venetsanopoulos | Below, we will describe all Synnefo APIs with conjuction to the OpenStack APIs. |
91 | 991d0507 | Constantinos Venetsanopoulos | |
92 | 991d0507 | Constantinos Venetsanopoulos | |
93 | 991d0507 | Constantinos Venetsanopoulos | Identity Service API (Astakos) |
94 | 991d0507 | Constantinos Venetsanopoulos | ============================== |
95 | 991d0507 | Constantinos Venetsanopoulos | |
96 | 991d0507 | Constantinos Venetsanopoulos | Currently, Astakos which is the Identity Management Service of Synnefo, has a |
97 | 991d0507 | Constantinos Venetsanopoulos | proprietary API, but we are moving to the OpenStack Keystone API. |
98 | 991d0507 | Constantinos Venetsanopoulos | |
99 | 991d0507 | Constantinos Venetsanopoulos | The current Identity Management API is: |
100 | 8f9976c6 | Constantinos Venetsanopoulos | |
101 | 8f9976c6 | Constantinos Venetsanopoulos | .. toctree:: |
102 | 8f9976c6 | Constantinos Venetsanopoulos | :maxdepth: 2 |
103 | 8f9976c6 | Constantinos Venetsanopoulos | |
104 | 991d0507 | Constantinos Venetsanopoulos | Identity API <astakos-api-guide> |
105 | 991d0507 | Constantinos Venetsanopoulos | |
106 | 8f9976c6 | Constantinos Venetsanopoulos | |
107 | 73ebcd68 | Constantinos Venetsanopoulos | Resource and Quota Service API (Astakos) |
108 | 73ebcd68 | Constantinos Venetsanopoulos | ======================================== |
109 | b32b5995 | Giorgos Korfiatis | |
110 | b32b5995 | Giorgos Korfiatis | .. toctree:: |
111 | b32b5995 | Giorgos Korfiatis | :maxdepth: 2 |
112 | b32b5995 | Giorgos Korfiatis | |
113 | b32b5995 | Giorgos Korfiatis | Resource and Quota API <quota-api-guide.rst> |
114 | b32b5995 | Giorgos Korfiatis | |
115 | bc055d09 | Constantinos Venetsanopoulos | |
116 | 991d0507 | Constantinos Venetsanopoulos | Compute Service API (Cyclades) |
117 | 991d0507 | Constantinos Venetsanopoulos | ============================== |
118 | bc055d09 | Constantinos Venetsanopoulos | |
119 | 991d0507 | Constantinos Venetsanopoulos | The Compute part of Cyclades exposes the OpenStack Compute API with minor |
120 | 991d0507 | Constantinos Venetsanopoulos | changes wherever needed. |
121 | 991d0507 | Constantinos Venetsanopoulos | |
122 | 991d0507 | Constantinos Venetsanopoulos | This is the Cyclades/Compute API: |
123 | bc055d09 | Constantinos Venetsanopoulos | |
124 | bc055d09 | Constantinos Venetsanopoulos | .. toctree:: |
125 | bc055d09 | Constantinos Venetsanopoulos | :maxdepth: 2 |
126 | bc055d09 | Constantinos Venetsanopoulos | |
127 | bc055d09 | Constantinos Venetsanopoulos | Compute API <cyclades-api-guide> |
128 | bc055d09 | Constantinos Venetsanopoulos | |
129 | bc055d09 | Constantinos Venetsanopoulos | |
130 | 991d0507 | Constantinos Venetsanopoulos | Network Service API (Cyclades) |
131 | 991d0507 | Constantinos Venetsanopoulos | ============================== |
132 | 991d0507 | Constantinos Venetsanopoulos | |
133 | 991d0507 | Constantinos Venetsanopoulos | The Network Service is implemented inside Cyclades. It exposes the part of the |
134 | 991d0507 | Constantinos Venetsanopoulos | OpenStack Compute API that has to do with Networks. The OpenStack Quantum API |
135 | 991d0507 | Constantinos Venetsanopoulos | is not implemented yet. |
136 | 991d0507 | Constantinos Venetsanopoulos | |
137 | 991d0507 | Constantinos Venetsanopoulos | Please consult the :ref:`Cyclades/Network API <cyclades-api-guide>` for more |
138 | 991d0507 | Constantinos Venetsanopoulos | details. |
139 | 1837aa54 | Giorgos Verigakis | |
140 | bc055d09 | Constantinos Venetsanopoulos | |
141 | 73ebcd68 | Constantinos Venetsanopoulos | Image Service API (Cyclades) |
142 | 73ebcd68 | Constantinos Venetsanopoulos | ============================ |
143 | bc055d09 | Constantinos Venetsanopoulos | |
144 | 73ebcd68 | Constantinos Venetsanopoulos | The Image Service is implemented inside Cyclades. It exposes the OpenStack |
145 | 73ebcd68 | Constantinos Venetsanopoulos | Glance API with minor changes wherever needed. |
146 | 991d0507 | Constantinos Venetsanopoulos | |
147 | 73ebcd68 | Constantinos Venetsanopoulos | This is the Cyclades/Image API: |
148 | 1837aa54 | Giorgos Verigakis | |
149 | 1837aa54 | Giorgos Verigakis | .. toctree:: |
150 | 1837aa54 | Giorgos Verigakis | :maxdepth: 2 |
151 | 1837aa54 | Giorgos Verigakis | |
152 | 1837aa54 | Giorgos Verigakis | Image API <plankton-api-guide> |
153 | 1837aa54 | Giorgos Verigakis | |
154 | bc055d09 | Constantinos Venetsanopoulos | |
155 | 991d0507 | Constantinos Venetsanopoulos | Storage Service API (Pithos) |
156 | 991d0507 | Constantinos Venetsanopoulos | ============================ |
157 | 991d0507 | Constantinos Venetsanopoulos | |
158 | 991d0507 | Constantinos Venetsanopoulos | Pithos is the Storage Service of Synnefo and it exposes the OpenStack Object |
159 | 991d0507 | Constantinos Venetsanopoulos | Storage API with extensions for advanced operations, e.g., syncing. |
160 | bc055d09 | Constantinos Venetsanopoulos | |
161 | 301294a9 | Constantinos Venetsanopoulos | This is the Pithos Object Storage API: |
162 | 8f9976c6 | Constantinos Venetsanopoulos | |
163 | 8f9976c6 | Constantinos Venetsanopoulos | .. toctree:: |
164 | 8f9976c6 | Constantinos Venetsanopoulos | :maxdepth: 2 |
165 | 8f9976c6 | Constantinos Venetsanopoulos | |
166 | dad708b4 | Antony Chazapis | Object Storage API <pithos-api-guide> |
167 | 8f9976c6 | Constantinos Venetsanopoulos | |
168 | 991d0507 | Constantinos Venetsanopoulos | |
169 | 8f9976c6 | Constantinos Venetsanopoulos | Implementing new clients |
170 | 8f9976c6 | Constantinos Venetsanopoulos | ======================== |
171 | 8f9976c6 | Constantinos Venetsanopoulos | |
172 | 8f9976c6 | Constantinos Venetsanopoulos | In this section we discuss implementation guidelines, that a developer should |
173 | 8f9976c6 | Constantinos Venetsanopoulos | take into account before writing his own client for the above APIs. Before, |
174 | 8f9976c6 | Constantinos Venetsanopoulos | starting your client implementation, make sure you have thoroughly read the |
175 | 8f9976c6 | Constantinos Venetsanopoulos | corresponding Synnefo API. |
176 | 8f9976c6 | Constantinos Venetsanopoulos | |
177 | 301294a9 | Constantinos Venetsanopoulos | Pithos clients |
178 | 301294a9 | Constantinos Venetsanopoulos | -------------- |
179 | 8f9976c6 | Constantinos Venetsanopoulos | |
180 | 8f9976c6 | Constantinos Venetsanopoulos | User Experience |
181 | 8f9976c6 | Constantinos Venetsanopoulos | ~~~~~~~~~~~~~~~ |
182 | 8f9976c6 | Constantinos Venetsanopoulos | |
183 | 8f9976c6 | Constantinos Venetsanopoulos | Hopefully this API will allow for a multitude of client implementations, each |
184 | 8f9976c6 | Constantinos Venetsanopoulos | supporting a different device or operating system. All clients will be able to |
185 | 8f9976c6 | Constantinos Venetsanopoulos | manipulate containers and objects - even software only designed for OOS API |
186 | 8f9976c6 | Constantinos Venetsanopoulos | compatibility. But a Pithos interface should not be only about showing |
187 | 8f9976c6 | Constantinos Venetsanopoulos | containers and folders. There are some extra user interface elements and |
188 | 8f9976c6 | Constantinos Venetsanopoulos | functionalities that should be common to all implementations. |
189 | 8f9976c6 | Constantinos Venetsanopoulos | |
190 | 8f9976c6 | Constantinos Venetsanopoulos | Upon entrance to the service, a user is presented with the following elements - |
191 | 8f9976c6 | Constantinos Venetsanopoulos | which can be represented as folders or with other related icons: |
192 | 8f9976c6 | Constantinos Venetsanopoulos | |
193 | 8f9976c6 | Constantinos Venetsanopoulos | * The ``home`` element, which is used as the default entry point to the user's |
194 | 8f9976c6 | Constantinos Venetsanopoulos | "files". Objects under ``home`` are represented in the usual hierarchical |
195 | 8f9976c6 | Constantinos Venetsanopoulos | organization of folders and files. |
196 | 8f9976c6 | Constantinos Venetsanopoulos | * The ``trash`` element, which contains files that have been marked for |
197 | 8f9976c6 | Constantinos Venetsanopoulos | deletion, but can still be recovered. |
198 | 8f9976c6 | Constantinos Venetsanopoulos | * The ``shared`` element, which contains all objects shared by the user to |
199 | 8f9976c6 | Constantinos Venetsanopoulos | other users of the system. |
200 | 8f9976c6 | Constantinos Venetsanopoulos | * The ``others`` element, which contains all objects that other users share |
201 | 8f9976c6 | Constantinos Venetsanopoulos | with the user. |
202 | 8f9976c6 | Constantinos Venetsanopoulos | * The ``groups`` element, which contains the names of groups the user has |
203 | 8f9976c6 | Constantinos Venetsanopoulos | defined. Each group consists of a user list. Group creation, deletion, and |
204 | 8f9976c6 | Constantinos Venetsanopoulos | manipulation is carried out by actions originating here. |
205 | 8f9976c6 | Constantinos Venetsanopoulos | * The ``history`` element, which allows browsing past instances of ``home`` |
206 | 8f9976c6 | Constantinos Venetsanopoulos | and - optionally - ``trash``. |
207 | 8f9976c6 | Constantinos Venetsanopoulos | |
208 | 991d0507 | Constantinos Venetsanopoulos | Objects in Pithos can be: |
209 | 8f9976c6 | Constantinos Venetsanopoulos | |
210 | 8f9976c6 | Constantinos Venetsanopoulos | * Moved to trash and then deleted. |
211 | 8f9976c6 | Constantinos Venetsanopoulos | * Shared with specific permissions. |
212 | 991d0507 | Constantinos Venetsanopoulos | * Made public (shared with non-Pithos users). |
213 | 8f9976c6 | Constantinos Venetsanopoulos | * Restored from previous versions. |
214 | 8f9976c6 | Constantinos Venetsanopoulos | |
215 | 8f9976c6 | Constantinos Venetsanopoulos | Some of these functions are performed by the client software and some by the |
216 | 991d0507 | Constantinos Venetsanopoulos | Pithos server. |
217 | 8f9976c6 | Constantinos Venetsanopoulos | |
218 | 8f9976c6 | Constantinos Venetsanopoulos | In the first version of Pithos, objects could also be assigned custom tags. |
219 | 8f9976c6 | Constantinos Venetsanopoulos | This is no longer supported. Existing deployments can migrate tags into a |
220 | 8f9976c6 | Constantinos Venetsanopoulos | specific metadata value, i.e. ``X-Object-Meta-Tags``. |
221 | 8f9976c6 | Constantinos Venetsanopoulos | |
222 | 8f9976c6 | Constantinos Venetsanopoulos | Implementation Guidelines |
223 | 8f9976c6 | Constantinos Venetsanopoulos | ~~~~~~~~~~~~~~~~~~~~~~~~~ |
224 | 8f9976c6 | Constantinos Venetsanopoulos | |
225 | 991d0507 | Constantinos Venetsanopoulos | Pithos clients should use the ``pithos`` and ``trash`` containers for active |
226 | 8f9976c6 | Constantinos Venetsanopoulos | and inactive objects respectively. If any of these containers is not found, the |
227 | 8f9976c6 | Constantinos Venetsanopoulos | client software should create it, without interrupting the user's workflow. The |
228 | 8f9976c6 | Constantinos Venetsanopoulos | ``home`` element corresponds to ``pithos`` and the ``trash`` element to |
229 | 8f9976c6 | Constantinos Venetsanopoulos | ``trash``. Use ``PUT`` with the ``X-Move-From`` header, or ``MOVE`` to transfer |
230 | 8f9976c6 | Constantinos Venetsanopoulos | objects from one container to the other. Use ``DELETE`` to remove from |
231 | 8f9976c6 | Constantinos Venetsanopoulos | ``pithos`` without trashing, or to remove from ``trash``. When moving objects, |
232 | 8f9976c6 | Constantinos Venetsanopoulos | detect naming conflicts with the ``If-Match`` or ``If-None-Match`` headers. |
233 | 8f9976c6 | Constantinos Venetsanopoulos | Such conflicts should be resolved by the user. |
234 | 8f9976c6 | Constantinos Venetsanopoulos | |
235 | 8f9976c6 | Constantinos Venetsanopoulos | Object names should use the ``/`` delimiter to impose a hierarchy of folders |
236 | 8f9976c6 | Constantinos Venetsanopoulos | and files. |
237 | 8f9976c6 | Constantinos Venetsanopoulos | |
238 | 8f9976c6 | Constantinos Venetsanopoulos | The ``shared`` element should be implemented as a read-only view of the |
239 | 8f9976c6 | Constantinos Venetsanopoulos | ``pithos`` container, using the ``shared`` parameter when listing objects. The |
240 | 8f9976c6 | Constantinos Venetsanopoulos | ``others`` element, should start with a top-level ``GET`` to retrieve the list |
241 | 8f9976c6 | Constantinos Venetsanopoulos | of accounts accessible to the user. It is suggested that the client software |
242 | 8f9976c6 | Constantinos Venetsanopoulos | hides the next step of navigation - the container - if it only includes |
243 | 8f9976c6 | Constantinos Venetsanopoulos | ``pithos`` and forwards the user directly to the objects. |
244 | 8f9976c6 | Constantinos Venetsanopoulos | |
245 | 8f9976c6 | Constantinos Venetsanopoulos | Public objects are not included in ``shared`` and ``others`` listings. It is |
246 | 8f9976c6 | Constantinos Venetsanopoulos | suggested that they are marked in a visually distinctive way in ``pithos`` |
247 | 8f9976c6 | Constantinos Venetsanopoulos | listings (for example using an icon overlay). |
248 | 8f9976c6 | Constantinos Venetsanopoulos | |
249 | 8f9976c6 | Constantinos Venetsanopoulos | A special application menu, or a section in application preferences, should be |
250 | 8f9976c6 | Constantinos Venetsanopoulos | devoted to managing groups (the ``groups`` element). All group-related actions |
251 | 8f9976c6 | Constantinos Venetsanopoulos | are implemented at the account level. |
252 | 8f9976c6 | Constantinos Venetsanopoulos | |
253 | 8f9976c6 | Constantinos Venetsanopoulos | Browsing past versions of objects should be available both at the object and |
254 | 8f9976c6 | Constantinos Venetsanopoulos | the container level. At the object level, a list of past versions can be |
255 | 8f9976c6 | Constantinos Venetsanopoulos | included in the screen showing details or more information on the object |
256 | 8f9976c6 | Constantinos Venetsanopoulos | (metadata, permissions, etc.). At the container level, it is suggested that |
257 | 8f9976c6 | Constantinos Venetsanopoulos | clients use a ``history`` element, which presents to the user a read-only, |
258 | 8f9976c6 | Constantinos Venetsanopoulos | time-variable view of ``pithos`` contents. This can be accomplished via the |
259 | 8f9976c6 | Constantinos Venetsanopoulos | ``until`` parameter in listings. Optionally, ``history`` may include ``trash``. |
260 | 8f9976c6 | Constantinos Venetsanopoulos | |
261 | 8f9976c6 | Constantinos Venetsanopoulos | Uploading and downloading data |
262 | 8f9976c6 | Constantinos Venetsanopoulos | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
263 | 8f9976c6 | Constantinos Venetsanopoulos | |
264 | 8f9976c6 | Constantinos Venetsanopoulos | By using hashmaps to upload and download objects the corresponding operations |
265 | 8f9976c6 | Constantinos Venetsanopoulos | can complete much faster. |
266 | 8f9976c6 | Constantinos Venetsanopoulos | |
267 | 8f9976c6 | Constantinos Venetsanopoulos | In the case of an upload, only the missing blocks will be submitted to the |
268 | 8f9976c6 | Constantinos Venetsanopoulos | server: |
269 | 8f9976c6 | Constantinos Venetsanopoulos | |
270 | 8f9976c6 | Constantinos Venetsanopoulos | * Calculate the hash value for each block of the object to be uploaded. Use |
271 | 8f9976c6 | Constantinos Venetsanopoulos | the hash algorithm and block size of the destination container. |
272 | 8f9976c6 | Constantinos Venetsanopoulos | * Send a hashmap ``PUT`` request for the object. |
273 | 8f9976c6 | Constantinos Venetsanopoulos | |
274 | 8f9976c6 | Constantinos Venetsanopoulos | * Server responds with status ``201`` (Created): |
275 | 8f9976c6 | Constantinos Venetsanopoulos | |
276 | 8f9976c6 | Constantinos Venetsanopoulos | * Blocks are already on the server. The object has been created. Done. |
277 | 8f9976c6 | Constantinos Venetsanopoulos | |
278 | 8f9976c6 | Constantinos Venetsanopoulos | * Server responds with status ``409`` (Conflict): |
279 | 8f9976c6 | Constantinos Venetsanopoulos | |
280 | 8f9976c6 | Constantinos Venetsanopoulos | * Server's response body contains the hashes of the blocks that do not |
281 | 8f9976c6 | Constantinos Venetsanopoulos | exist on the server. |
282 | 8f9976c6 | Constantinos Venetsanopoulos | * For each hash value in the server's response (or all hashes together): |
283 | 8f9976c6 | Constantinos Venetsanopoulos | |
284 | 8f9976c6 | Constantinos Venetsanopoulos | * Send a ``POST`` request to the destination container with the |
285 | 8f9976c6 | Constantinos Venetsanopoulos | corresponding data. |
286 | 8f9976c6 | Constantinos Venetsanopoulos | |
287 | 8f9976c6 | Constantinos Venetsanopoulos | * Repeat hashmap ``PUT``. Fail if the server's response is not ``201``. |
288 | 8f9976c6 | Constantinos Venetsanopoulos | |
289 | 8f9976c6 | Constantinos Venetsanopoulos | Consulting hashmaps when downloading allows for resuming partially transferred |
290 | 8f9976c6 | Constantinos Venetsanopoulos | objects. The client should retrieve the hashmap from the server and compare it |
291 | 8f9976c6 | Constantinos Venetsanopoulos | with the hashmap computed from the respective local file. Any missing parts can |
292 | 8f9976c6 | Constantinos Venetsanopoulos | be downloaded with ``GET`` requests with the additional ``Range`` header. |
293 | 8f9976c6 | Constantinos Venetsanopoulos | |
294 | 8f9976c6 | Constantinos Venetsanopoulos | Syncing |
295 | 8f9976c6 | Constantinos Venetsanopoulos | ~~~~~~~ |
296 | 8f9976c6 | Constantinos Venetsanopoulos | |
297 | 8f9976c6 | Constantinos Venetsanopoulos | Consider the following algorithm for synchronizing a local folder with the |
298 | 8f9976c6 | Constantinos Venetsanopoulos | server. The "state" is the complete object listing, with the corresponding |
299 | 8f9976c6 | Constantinos Venetsanopoulos | attributes. |
300 | 114b028e | marioskogias | |
301 | 30aae6bf | Giorgos Verigakis | .. code-block:: python |
302 | 30aae6bf | Giorgos Verigakis | |
303 | 30aae6bf | Giorgos Verigakis | # L: Local State, the last synced state of the object. |
304 | 30aae6bf | Giorgos Verigakis | # Stored locally (e.g. in an SQLite database) |
305 | 114b028e | marioskogias | |
306 | 30aae6bf | Giorgos Verigakis | # C: Current State, the current local state of the object |
307 | 30aae6bf | Giorgos Verigakis | # Returned by the filesystem |
308 | 114b028e | marioskogias | |
309 | 30aae6bf | Giorgos Verigakis | # S: Server State, the current server state of the object |
310 | 30aae6bf | Giorgos Verigakis | # Returned by the server (HTTP request) |
311 | 114b028e | marioskogias | |
312 | 30aae6bf | Giorgos Verigakis | def sync(path): |
313 | 30aae6bf | Giorgos Verigakis | L = get_local_state(path) # Database action |
314 | 30aae6bf | Giorgos Verigakis | C = get_current_state(path) # Filesystem action |
315 | 30aae6bf | Giorgos Verigakis | S = get_server_state(path) # Network action |
316 | 114b028e | marioskogias | |
317 | 30aae6bf | Giorgos Verigakis | if C == L: |
318 | 30aae6bf | Giorgos Verigakis | # No local changes |
319 | 30aae6bf | Giorgos Verigakis | if S == L: |
320 | 30aae6bf | Giorgos Verigakis | # No remote changes, nothing to do |
321 | 30aae6bf | Giorgos Verigakis | return |
322 | 30aae6bf | Giorgos Verigakis | else: |
323 | 30aae6bf | Giorgos Verigakis | # Update local state to match that of the server |
324 | 30aae6bf | Giorgos Verigakis | download(path) |
325 | 30aae6bf | Giorgos Verigakis | update_local_state(path, S) |
326 | 8f9976c6 | Constantinos Venetsanopoulos | else: |
327 | 30aae6bf | Giorgos Verigakis | # Local changes exist |
328 | 30aae6bf | Giorgos Verigakis | if S == L: |
329 | 30aae6bf | Giorgos Verigakis | # No remote changes, update the server and the local state |
330 | 30aae6bf | Giorgos Verigakis | upload(path) |
331 | 30aae6bf | Giorgos Verigakis | update_local_state(path, C) |
332 | 8f9976c6 | Constantinos Venetsanopoulos | else: |
333 | 30aae6bf | Giorgos Verigakis | # Both local and server changes exist |
334 | 30aae6bf | Giorgos Verigakis | if C == S: |
335 | 30aae6bf | Giorgos Verigakis | # We were lucky, both did the same |
336 | 30aae6bf | Giorgos Verigakis | update_local_state(path, C) |
337 | 30aae6bf | Giorgos Verigakis | else: |
338 | 30aae6bf | Giorgos Verigakis | # Conflicting changes exist |
339 | 30aae6bf | Giorgos Verigakis | conflict() |
340 | 114b028e | marioskogias | |
341 | 8f9976c6 | Constantinos Venetsanopoulos | |
342 | 8f9976c6 | Constantinos Venetsanopoulos | Notes: |
343 | 8f9976c6 | Constantinos Venetsanopoulos | |
344 | 8f9976c6 | Constantinos Venetsanopoulos | * States represent file hashes (it is suggested to use Merkle). Deleted or |
345 | 8f9976c6 | Constantinos Venetsanopoulos | non-existing files are assumed to have a magic hash (e.g. empty string). |