/doc/arch/aquarium.tex - Diff - Aquarium - Greek Research and Technology Network's projects

Revision 44840f76 doc/arch/aquarium.tex

     \maketitle
     \begin{abstract}
         This paper describes the architecture for the Aquarium cloud infrastructure
         software.
         software. Aquarium is a new software we have built whose main function is to associate cloud resources usage with charging policies.
     \end{abstract}
     \category{D.3.3}{Programming Languages}{Language Constructs and Features}[Control structures]
-...
     failure, Aquarium must not corrupt any billing data under any circumstances,
     while it should reach an operating state very fast after a service restart.
     \section{Architecture}
     \input{arch}
     \section{Domain Modeling}
     \subsection{Basic terminology}
     We have already mentioned several entities in our description so far. Let us be a bit more specific on several key terms.
     \section{Implementation}
     \begin{description}
     \item[Credits]
     The analog of money. Credits are the `universal money` within Aquarium.
     \subsection{The configuration DSL}
     \label{sec:dsl}
     The configuration requirements presented above were addressed by creating a new
     domain specific language ({\sc dsl}), based on the YAML format.  The DSL
     enables administrators to specify billable resources, billing policies and
     price lists and combine them arbitrarily into agreements applicable to specific
     users, user groups or the whole system.
     The DSL supports inheritance for policies, price lists and agreements and composition in the case of agreements.
     It also facilitates the
     definition of generic, repeatable debiting rules, which are then used by the
     system to refill the user's account with credits on a periodic based.
     \item[Resource]
     A billable/chargeable entity. We generally need credits to use a resource. When a resource is used,  then consume credits. Examples of resources are the `download bandwidth` and, respectively,  the 'upload bandwidth', the `disk space` and the `VM time` to name a few. Generally speaking, the ``resource'' term specifies the type. A resource may have several properties attached, i.e. it name, unit of measure, a description of how its consumption translates to credit and whether it can have more than one instances.
     The DSL is in itself based on five top-level entities, namely:
     \item[Resource instance]
     A user may have several instances of a resource type. For example, regarding a ``Virtual Machine'' resource, a user may have more then one of them. They are distinguished by their unique resource instance identifier. We call resources that can have more than one instance ``complex'' resources.
     \begin{description}
     \item[Resource event]
     An event that is generated by a system, which is responsible for the resource.
     The resource event describes a state change for the resource. In particular, a resource event records the time when that state change occurred (`occurredMillis` attribute) and the changed value (`value` attribute).
         \item[Resources] specify the properties of resources that Aquarium knows
             about. Apart from the expected ones (name, unit etc), a resource has
             two properties that affect billing: \textsf{costpolicy} defines the
             algorithm to be used to calculate the resource usage, while the
             \textsf{complex} attribute defines whether a resource can have one or
             many instances per user.
     \item[Cost policy]
     A cost policy refers to a resource and it is the policy used in order to charge credits for resource usage. Cost policies come in three flavors, namely \textsf{continuous}, \textsf{discrete} and \textsf{onoff}.
     \item[Pricelists] assign a price tag to each resource, within a timeframe.
         \item[Pricelists] assign a price tag to each resource, within a timeframe.
         \item[Algorithms] specify the way the billing operation is done in response
             to a billing event. The simplest (and default) way is to multiply the
             billable quantity with the applicable price. To enable more complex billing
             scenarios, the Aquarium DSL supports a simple imperative language with
     \item[Charging algorithms] specify the way a resource event can generate consumed credits.
     A charging algorithm can be as simple as a direct multiplication of the
             chargeable resource quantity with the applicable price. We offer the option, though, to define more complex charging  scenarios, the Aquarium DSL supports a simple imperative language with
             a number of implicit variables (e.g. \texttt{price, volume, date})
             that enable administrators to specify, e.g. billing algorithms that
             scale with billable volume. Similarily to pricelists, algorithms
             scale with billable volume. Similarily to price lists, charging algorithms
             have an applicability timeframe attached to them.
         \item[Crediplans] define a number of credits to give to users and a repetition
     \item[Credit plans] define a number of credits to give to users and a repetition
             period.
         \item[Agreements] assign a name to algorithm, pricelist and creditplan triplets,
     \item[Agreements] assign a name to a charging algorithm, pricelist and creditplan triplets,
             which is then assigned to each user.
     \item[Resource instance state]
     A value which is associated with a resource instance. Usually a floating point number, as in the $10.5$ MB designation regarding a possible ``total current downloading bandwidth''.
     \item[User]
     An owner of resources and credits. Users are defined externally of Aquarium.
     \item[Resource event store]
     A datatabase, in the general sense, where resource events are stored.
     \item[User bill]
     A, usually, periodic statement of how many credits the user has consumed. It may contain detailed analysis that relates consumed credits to resources.
     \item[Billing period]
     A time period at the end of which we issue a user bill.
     A billing period is made of a starting date and a duration that is a multiple of a week.
     A usual billing period starts on a particular month date (eg. 3rd) and lasts for a month.
     Each resource type designates what happens to its accumulated value (if any) at the beginning of the billing period. Usually, at the beginning of the billing period, the accumulating amounts of resources are set accumulating amount. For example, for a monthly billing period, the total uploading bandwidth is reset to zero every month.
     \item[User state]
     The user state is made of the following distinct parts, of which the first two can be integrated to a unifying ``resource'' concept.
     \begin{enumerate}
     \item User credit state, that is the total credit amount for the user.
     \item User resource state, which refers to the state of each resource instance that the user owns.
     \item Processing state \TODO
     \end{enumerate}
     \item[Resource event processing]
     The set of algorithmic steps by which a resource event leads to state changes of the user state.
     \end{description}
     \subsection{The configuration DSL}
     \label{sec:dsl}
     The configuration requirements of Aquarium  were addressed by creating a new
     domain specific language ({\sc dsl}), based on the YAML format.  The DSL
     enables administrators to specify chargeable resources, charging policies and
     price lists and combine them arbitrarily into agreements applicable to specific
     users, user groups or the whole system.
     The DSL supports inheritance for policies, price lists and agreements and composition in the case of agreements.
     It also facilitates the
     definition of generic, repeatable debiting rules, which are then used by the
     system to refill the user's account with credits on a periodic base.
     From the previously specified term, the following five are used in the DSL:
     \begin{enumerate}
     \item Resources
     \item Charging algorithms
     \item Pricelists
     \item Credit plans
     \item Agreements
     \end{enumerate}
     \begin{figure}
     \lstset{language=c, basicstyle=\footnotesize,
     stringstyle=\ttfamily,
-...
     the user's balance, Aquarium performs account updates asynchronously and
     concurrently for all users.
     Before delving into the details of what things Aquarium computes and how these computations are performed, some terminology is needed. The following are the basic entities  involved.
     \begin{description}
     \item[Credits]
     The analog of money. Credits are the `universal money` within Aquarium.
     \item[Resource]
     A billable/chargeable entity. We generally need credits to use a resource. When a resource is used,  then consume credits. Examples of resources are the `download bandwidth` and, respectively,  the 'upload bandwidth', the `disk space` and the `VM time` to name a few. Generally speaking, the ``resource'' term specifies the type.
     \item[Resource instance]
     A user may have several instances of a resource type. For example, regarding a ``Virtual Machine'' resource, a user may have more then one of them. They are distinguished by their unique resource instance identifier.
     \item[Resource instance state]
     A value which is associated with a resource instance. Usually a floating point number, as in the $10.5$ MB designation regarding a possible ``total current downloading bandwidth''.
     \item[User]
     An owner of resources and credits. Users are defined externally of Aquarium.
     \item[Resource event]
     An event that is generated by a system, which is responsible for the resource.
     The resource event describes a state change for the resource. In particular, a resource event records the time when that state change occurred (`occurredMillis` attribute) and the changed value (`value` attribute).
     \item[Resource event store]
     A datatabase, in the general sense, where resource events are stored.
     \item[Cost policy]
     A cost policy refers to a resource and it is the policy used in order to charge credits for resource usage.
     \item[User bill]
     A, usually, periodic statement of how many credits the user has consumed. It may contain detailed analysis that relates consumed credits to resources.
     \item[Billing period]
     A time period at the end of which we issue a user bill.
     A billing period is made of a starting date and a duration that is a multiple of a week.
     A usual billing period starts on a particular month date (eg. 3rd) and lasts for a month.
     Each resource type designates what happens to its accumulated value (if any) at the beginning of the billing period. Usually, at the beginning of the billing period, the accumulating amounts of resources are set accumulating amount. For example, for a monthly billing period, the total uploading bandwidth is reset to zero every month.
     \item[User state]
     The user state is made of the following distinct parts, of which the first two can be integrated to a unifying ``resource'' concept.
     \begin{enumerate}
     \item User credit state, that is the total credit amount for the user.
     \item User resource state, which refers to the state of each resource instance that the user owns.
     \item Processing state \TODO
     \end{enumerate}
     \item[Resource event processing]
     The set of algorithmic steps by which a resource event leads to state changes of the user state.
     \item[]
     \item[]
     \item[]
     \item[]
     \item[]
     \end{description}
     Per resource, the charging operation is affected by the cost policy and complexity
     parameters. Specifically, the 3 available cost policies affect the calculation
-...
     \subsection{User State}
     \label{sec:ustate}
     \section{Architecture}
     \input{arch}
     \section{Performance}
     To evaluate the performance and scalability of Aquarium, we performed two

Also available in: Unified diff