Automatically cleanup _temporary_ids at save

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Separate the computation of all config IDs

We will need this in another place, so we abstract the 'compute all
current IDs' functionality into a separate function. We also change the
name of the _ComputeAllLVs to _AllLVs to match the other _All*s
functions.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Node init: copy hmac key as well

Without this confd will not start when a node is added to the cluster.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unpack the confd reply as an object, from the dict

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Install guide: talk about routing mode

The networking section is updated to talk about both routing and
bridging.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Update hypervisor installation section

Currently the section is called "Installing Xen" and also mentions KVM.
Updating the title and also make it slightly less xen-centric.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

document the nic link meaning in routed mode

Currently this feature is hypervisor specific, but under kvm we use it
by default (aka: if the user hasn't specified another network script
which overrides our auto-generated one) to specify the routing table.
This was already hinted as a possibility in the manpages, but never
explicitely stated as a reality, so we update gnt-cluster(8) to actually
say this, and gnt-{backup,instance}(8) to point to gnt-cluster(8) for
more information.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

KVM nic script: enable interface forwarding

If forwarding is enabled globally this is a no-op. If instead it's
enabled only for some special interfaces where instance traffic has to
go to/comes from (for example a gre tunnel) then it's useful to
explicitely enable it for the instances interfaces as well.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

KVM nic script: use routed link as table

In order to be able to maintain the node network standard routing
untouched while routing instance traffic through a different dedicated
interface (eg: a gre tunnel) we need to specify the instance routing
path inside a separate table, which will also contain different default
gateway values. This patch fixes the kvm default network script to use
the nic link, if present, as a routing table to use instead of the
default one.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Confd client library: enable optional logging

If a logger is passed in, we log some debugging messages that might help
someone who's debugging a confd client to understand what's going on.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Confd client: make confd port configurable

The port can be now chosen at library init time, with a default of
calling GetDaemonPort.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Confd client: Change callback model

We move to one callback in total, rather than one per call, and call it
both for server replies and request expiring.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Confd client: make SendRequest args optional

By default "None" will be used as an args value

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Change config upgrade to be explicit

Currently the config upgrade is done at each object instantiation, that
means that ganeti-noded will run UpgradeConfig on all objects received
remotely (instances, disks, nics). This is not so good, so this patch
changes it so that only the ConfigWriter runs this method at
configuration load time.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Confd: add instances IPs query

Extend confd to answer queries about instances IPs.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Fix utils.MatchNameComponent for full matches

While ‘test1’ matches both ‘test1’ and ‘test1.example’, it has a full,
exact match and we should return it if that is the case.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Design doc changes for UUID tracking of objects

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Fix _RemoveDisk for file based instances

During 621b7678 two typos were introduced which prevent file based
instances removal to work correctly. Fixing both of them to what they
were meant to be.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Re-wrap locking changes design to 76 chars per line

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Revert "Include NEWS file in documentation"

This reverts commit 350ecfecca136f50877898774c6b40014b5c2b1c. It broke
distcheck.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

gnt-instance info: fix nic display

In branch-2.1 the conversion of NICs to Tuples has been unified to the
_NICListToTuple function, which is used by both hook generation and
instance info generation. Unfortunately the ip and mac fields got
swapped from what instance info expected, so the mac is shown as ip, and
vice versa. Fixing this.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

cmdlib._CreateDisks fix a broken result.Raise

The format string has the ": %s" at the end, but no argument is passed,
which of course raises a TypeError. Removing ": %s" as it's added by
the RpcResult Raise() method anyway.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Include NEWS file in documentation

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Convert NEWS file to RST

- Add markup for titles
- Convert quotes
- Reindent and rewrap to 72 chars per line

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Unify the instance creation code

Currently the AddInstance in gnt-instance and ImportInstance in
gnt-backup duplicate all of their code except the actual opcode creation
(the parameters to it). By moving this to cli.py (not optimal location,
but we don't have another one), we can use a single copy of the code,
and simply change based on the mode parameter the few different
opcode parameters.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify multi-instance handling in gnt-instance

Start, stop and reboot have the same code for dealing with
multi-instance handling. This patch moves all that into a single generic
function, and leaves only the building of the specific opcode for the
operation in the individual functions.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Fix gnt-instance start/stop/… args

The gnt-instance commands that have the multi-filter can take also zero
arguments, if the --all flag is passed. But they currently are declared
with min=1, and this is a regression due to the recent bash completion
changes.

There is also the problem that --primary/--secondary expect a node and
not an instance, but currently the completion doesn't support this
distinction.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Remove explicit DEBUG_OPT and add it by default

Since >90% of the commands take the “--debug” option, and all should
actually take it (the gnt-job command is currently missing it), it makes
sense to simply remove this and add it by default in cli.py.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Change indentation style in commands definitions

Currently, the indentation level for a command's definition depends on
the command name length, and this makes the commands dict unpleasant to
the eye. This patch changes it so that a new line begins right after the
left parenthesis starting the command tuple.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Make ganeti-watcher use the standard debug option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--no-shutdown” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--ignore-secondaries” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--reboot-type” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move cluster verify options to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move more cluster init opts to cli.py

Some of these should be added to cluster modify too.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--mac-prefix” option to cli.py

The default has also been changed to None as this will be useful once
cluster modify will be able to change this setting too.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--no-voting” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--yes-do-it” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--vg-name” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--candidate-pool-size” option

This required a slight tweak since in init we want a different default.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--nic-parameters” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--enabled-hypervisors” option

This also required a change, as one of the cases used a different
default.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--no-lvm-storage” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--allocatable” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the node flags options to cli.py

This also unifies a little the parameters to the respective options.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--no-ssh-key-check” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--readd” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--secondary-ip” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--src-…” options to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--ignore-size” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--auto” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--on-secondary” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--on-primary” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--new-secondary” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--ignore-failures” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--select-os” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--all” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--static” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--cleanup” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--show-cmd” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move the “--no-start” option to cli.py

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the last “--node” (single node) option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--node” (as list of nodes) option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--node” (as instance placement) option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--non-live” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--ignore-consistency” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--os-size” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--disks” (disk indices) option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--disk” (identkeyval) option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--net” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--no-ip-check” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the last “--hypervisor” (as list) option

This third form takes a list of name+options, and is used only in
gnt-cluster. Who knew that we have three forms of ‘-H’?

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--hypervisor” (with name) option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the “--hypervisor” (params only) option

This unifies the two uses of --hypervisor-parameters with only options
(a=b,…) form. The patch changes a little SetInstanceParms since that
expected opts.hypervisor instead of opts.hvparams.

Note that there is another form of --hypervisor-parameters which also
takes the name, that comes in a separate patch.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Unify the “--backend-parameters” option

This is not straight-forward. First, gnt-cluster had
“--backend-parameters” wheres the rest of the script had just
“--backend”. I settled on the full form, since one can always abbreviate
options, but not vice-versa.

Second, the help in some cases was different, e.g. start instance
had “Temporary backend parameters” and modify had “Change backend
parameters”. I think this kind of distinctions are better served in the
man page. Alternatively, if we could copy options easily, one could
re-instantiate them with a changed help line.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Unify the “--os-type” option

This was already unified inside gnt-instance, I just moved that
definition (and upper-cased it) into cli.py.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Unify the “--iallocator” option

This patch also changes the gnt-debug's “--algorithm” into
IALLOCATOR_OPT, since it was the only non-standard use of this (due to
historical reasons). Since it's just a debug option,
backwards-compatibility is not critical here.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Unify the “--file-driver” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Unify the “--file-storage-dir” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Unify the “--no-nics” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Unify the “--disk-template” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Unify the “--no-wait-for-sync” option

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Explain the reason for fqdn hostnames

The install guide asks the user to use the fqdn in the hostname, but
doesn't explain why. This has been asked multiple times, so adding an
explanation there.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

Remove RpcResult.RemoteFailMsg completely

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add locking improvements to Ganeti 2.1 design document

Also increase the table of contents' depth to 4.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Add an error-simulation mode to cluster verify

One of the issues we have in ganeti is that it's very hard to test the
error-handling paths; QA and burnin only test the OK code-path, since
it's hard to simulate errors.

LUVerifyCluster is special amongst the LUs in the fact that a) it has a
lot of error paths and b) the error paths only log the error, they don't
do any rollback or other similar actions. Thus, it's enough for this LU
to separate the testing of the error condition from the logging of the
error condition.

This patch does this by replacing code blocks of the form:

  if x:
    log_error()
    [y]

into:

  log_error_if(x)
  [if x:
    y
  ]

After this change, it's simple enough to turn on logging of all errors
by adding a special case inside log_error_if such that if the incoming
opcode has a special ‘debug_simulate_errors’ attribute and it's true, it
will log unconditionally the error.

Surprisingly this also turns into an absolute code reduction, since some
of the if blocks were simplified. The only downside to this patch is
that the various _VerifyX() functions are now stateful (modifying an
attribute on the LU instance) instead of returning a boolean result.

Last note: yes, this discovered some error cases in the logging.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Introduce parseable error codes in LUVerifyCluster

Currently the output of cluster verify can be parsed for 'ERROR'
messages, but that is the only indication we get (error or no error). In
order to allow monitoring tools to separate different error conditions,
this patch introduces a new output format (“gnt-cluster verify
--error-codes”) that changes the output from human-friendly to
machine-friendly. In this mode, an error line changes from:
  ERROR: node node1: drbd minor 1 of instance inst1.is not active

to:
  ERROR:ENODEDRBD:node:node1:drbd minor 1 of instance inst1 is not active

i.e. the error message is a ‘:’-separated field, with ERROR in the first
place, the error code in the second, the object type (cluster, node,
instance) in the third, the name of the object (for nodes/instances) in
the fourth, and then the text message.

The patch also removes some of the verbosity of the operation
(“Verifying instance X”, “Verifying node X”) since on big clusters these
informational messages can quickly fill up an entire screen. The
original behaviour can be restored via the ‘--verbose’ option.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Unify the multiple 'verbose' options

Currently multiple files define a 'verbose' option. This patch moves all
these definitions to a single VERBOSE_OPT option which is exported from
cli.py.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Rewrite the lib/cli.py export list

Right now, the export list is a mess, it's not sorted, and the exported
items are in random places. This patches does grow the length of the
list, but at least it's more clear.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Show "--help" in bash completion

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Remove RpcResult.failed attribute

It's redundant with RpcResult.fail_msg. This change survived QA, burnin and
unittests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Confd: add primary IPs queries

Extend confd to answer queries about nodes primary IPs and master candidates
primary IPs.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

ganeti-confd: remove partial imports

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Confd client library

Initial confd client library implementation.  This initial version uses
asyncore, and supports answers via a callback.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Move fourcc packing/unpacking to main confd module

This way it can be used by the client as well

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Add unittests for confd constants

Checking that the fourcc code is indeed a fourcc code, and that request
codes and reply statuses are distinct.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

ganeti-confd: explicitely log failed big sends

Make sure that if we try to send packages which are too big (which
shouldn't happen) this gets properly logged in the config file.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

Add a magic fourcc code to confd packets

This will make it easier to change the protocol later on

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

AsyncUDPSocket: Move to a well defined UDP size

Currently we read maximum 4K packets, and don't check packets when
sending them. With this patch we move to a well defined maximum size of
60K.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

ConfdAsyncUDPServer: fix a docstring

It refers to an older input variable

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>