+News
+====
+
+Version 2.2.0 rc0
+-----------------
+
+*(UNRELEASED)*
+
+Important change: the internal RPC mechanism between Ganeti nodes has
+changed from using a home-grown http library (based on the Python base
+libraries) to use the PycURL library. This requires that PycURL is
+installed on nodes. Please note that on Debian/Ubuntu, PycURL is linked
+against GnuTLS by default. cURL's support for GnuTLS had known issues
+before cURL 7.21.0 and we recommend using the latest cURL release or
+linking against OpenSSL. Most other distributions already link PycURL
+and cURL against OpenSSL. The command::
+
+ python -c 'import pycurl; print pycurl.version'
+
+can be used to determine the libraries PycURL and cURL are linked
+against.
+
+Other significant changes:
+
+- Rewrote much of the internals of the job queue, in order to achieve
+ better parallelism; this decouples job query operations from the job
+ processing, and it should allow much nicer behaviour of the master
+ daemon under load, and it also has uncovered some long-standing bugs
+ related to the job serialisation (now fixed)
+- Added a default iallocator setting to the cluster parameters,
+ eliminating the need to always pass nodes or an iallocator for
+ operations that require selection of new node(s)
+- Added experimental support for the LXC virtualization method
+- Added support for OS parameters, which allows the installation of
+ instances to pass parameter to OS scripts in order to customise the
+ instance
+- Added a hypervisor parameter controlling the migration type (live or
+ non-live), since hypervisors have various levels of reliability; this
+ has renamed the 'live' parameter to 'mode'
+- Added a cluster parameter ``reserved_lvs`` that denotes reserved
+ logical volumes, meaning that cluster verify will ignore them and not
+ flag their presence as errors
+- The watcher will now reset the error count for failed instances after
+ 8 hours, thus allowing self-healing if the problem that caused the
+ instances to be down/fail to start has cleared in the meantime
+- Added a cluster parameter ``drbd_usermode_helper`` that makes Ganeti
+ check for, and warn, if the drbd module parameter ``usermode_helper``
+ is not consistent with the cluster-wide setting; this is needed to
+ make diagnose easier of failed drbd creations
+- Started adding base IPv6 support, but this is not yet
+ enabled/available for use
+- Rename operations (cluster, instance) will now return the new name,
+ which is especially useful if a short name was passed in
+- Added support for instance migration in RAPI
+- Added a tool to pre-configure nodes for the SSH setup, before joining
+ them to the cluster; this will allow in the future a simplified model
+ for node joining (but not yet fully enabled in 2.2); this needs the
+ paramiko python library
+- Fixed handling of name-resolving errors
+- Fixed consistency of job results on the error path
+- Fixed master-failover race condition when executed multiple times in
+ sequence
+- Fixed many bugs related to the job queue (mostly introduced during the
+ 2.2 development cycle, so not all are impacting 2.1)
+- Fixed instance migration with missing disk symlinks
+- Fixed handling of unknown jobs in ``gnt-job archive``
+- And many other small fixes/improvements
+
+Internal changes:
+
+- Enhanced both the unittest and the QA coverage
+- Switched the opcode validation to a generic model, and extended the
+ validation to all opcode parameters
+- Changed more parts of the code that write shell scripts to use the
+ same class for this
+- Switched the master daemon to use the asyncore library for the Luxi
+ server endpoint
+
+
+Version 2.2.0 beta 0
+--------------------
+
+*(Released Thu, 17 Jun 2010)*
+
+- Added tool (``move-instance``) and infrastructure to move instances
+ between separate clusters (see :doc:`separate documentation
+ <move-instance>` and :doc:`design document <design-2.2>`)
+- Added per-request RPC timeout
+- RAPI now requires a Content-Type header for requests with a body (e.g.
+ ``PUT`` or ``POST``) which must be set to ``application/json`` (see
+ RFC2616 (HTTP/1.1), section 7.2.1)
+- ``ganeti-watcher`` attempts to restart ``ganeti-rapi`` if RAPI is not
+ reachable
+- Implemented initial support for running Ganeti daemons as separate
+ users, see configure-time flags ``--with-user-prefix`` and
+ ``--with-group-prefix`` (only ``ganeti-rapi`` is supported at this
+ time)
+- Instances can be removed after export (``gnt-backup export
+ --remove-instance``)
+- Self-signed certificates generated by Ganeti now use a 2048 bit RSA
+ key (instead of 1024 bit)
+- Added new cluster configuration file for cluster domain secret
+- Import/export now use SSL instead of SSH
+- Added support for showing estimated time when exporting an instance,
+ see the ``ganeti-os-interface(7)`` manpage and look for
+ ``EXP_SIZE_FD``
+
+
+Version 2.1.6
+-------------
+
+*(Released Fri, 16 Jul 2010)*
+
+Bugfixes only:
+ - Add an option to only select some reboot types during qa/burnin.
+ (on some hypervisors consequent reboots are not supported)
+ - Fix infrequent race condition in master failover. Sometimes the old
+ master ip address would be still detected as up for a short time
+ after it was removed, causing failover to fail.
+ - Decrease mlockall warnings when the ctypes module is missing. On
+ Python 2.4 we support running even if no ctypes module is installed,
+ but we were too verbose about this issue.
+ - Fix building on old distributions, on which man doesn't have a
+ --warnings option.
+ - Fix RAPI not to ignore the MAC address on instance creation
+ - Implement the old instance creation format in the RAPI client.
+
+
+Version 2.1.5
+-------------
+
+*(Released Thu, 01 Jul 2010)*
+
+A small bugfix release:
+ - Fix disk adoption: broken by strict --disk option checking in 2.1.4
+ - Fix batch-create: broken in the whole 2.1 series due to a lookup on
+ a non-existing option
+ - Fix instance create: the --force-variant option was ignored
+ - Improve pylint 0.21 compatibility and warnings with Python 2.6
+ - Fix modify node storage with non-FQDN arguments
+ - Fix RAPI client to authenticate under Python 2.6 when used
+ for more than 5 requests needing authentication
+ - Fix gnt-instance modify -t (storage) giving a wrong error message
+ when converting a non-shutdown drbd instance to plain
+
+
+Version 2.1.4
+-------------
+
+*(Released Fri, 18 Jun 2010)*
+
+A small bugfix release:
+
+ - Fix live migration of KVM instances started with older Ganeti
+ versions which had fewer hypervisor parameters
+ - Fix gnt-instance grow-disk on down instances
+ - Fix an error-reporting bug during instance migration
+ - Better checking of the ``--net`` and ``--disk`` values, to avoid
+ silently ignoring broken ones
+ - Fix an RPC error reporting bug affecting, for example, RAPI client
+ users
+ - Fix bug triggered by different API version os-es on different nodes
+ - Fix a bug in instance startup with custom hvparams: OS level
+ parameters would fail to be applied.
+ - Fix the RAPI client under Python 2.6 (but more work is needed to
+ make it work completely well with OpenSSL)
+ - Fix handling of errors when resolving names from DNS
+
+
+Version 2.1.3
+-------------
+
+*(Released Thu, 3 Jun 2010)*
+
+A medium sized development cycle. Some new features, and some
+fixes/small improvements/cleanups.
+
+Significant features
+~~~~~~~~~~~~~~~~~~~~
+
+The node deamon now tries to mlock itself into memory, unless the
+``--no-mlock`` flag is passed. It also doesn't fail if it can't write
+its logs, and falls back to console logging. This allows emergency
+features such as ``gnt-node powercycle`` to work even in the event of a
+broken node disk (tested offlining the disk hosting the node's
+filesystem and dropping its memory caches; don't try this at home)
+
+KVM: add vhost-net acceleration support. It can be tested with a new
+enough version of the kernel and of qemu-kvm.
+
+KVM: Add instance chrooting feature. If you use privilege dropping for
+your VMs you can also now force them to chroot to an empty directory,
+before starting the emulated guest.
+
+KVM: Add maximum migration bandwith and maximum downtime tweaking
+support (requires a new-enough version of qemu-kvm).
+
+Cluster verify will now warn if the master node doesn't have the master
+ip configured on it.
+
+Add a new (incompatible) instance creation request format to RAPI which
+supports all parameters (previously only a subset was supported, and it
+wasn't possible to extend the old format to accomodate all the new
+features. The old format is still supported, and a client can check for
+this feature, before using it, by checking for its presence in the
+``features`` RAPI resource.
+
+Now with ancient latin support. Try it passing the ``--roman`` option to
+``gnt-instance info``, ``gnt-cluster info`` or ``gnt-node list``
+(requires the python-roman module to be installed, in order to work).
+
+Other changes
+~~~~~~~~~~~~~
+
+As usual many internal code refactorings, documentation updates, and
+such. Among others:
+
+ - Lots of improvements and cleanups to the experimental Remote API
+ (RAPI) client library.
+ - A new unit test suite for the core daemon libraries.
+ - A fix to creating missing directories makes sure the umask is not
+ applied anymore. This enforces the same directory permissions
+ everywhere.
+ - Better handling terminating daemons with ctrl+c (used when running
+ them in debugging mode).
+ - Fix a race condition in live migrating a KVM instance, when stat()
+ on the old proc status file returned EINVAL, which is an unexpected
+ value.
+ - Fixed manpage checking with newer man and utf-8 charachters. But now
+ you need the en_US.UTF-8 locale enabled to build Ganeti from git.
+
+
+Version 2.1.2.1
+---------------
+
+*(Released Fri, 7 May 2010)*
+
+Fix a bug which prevented untagged KVM instances from starting.
+
+
+Version 2.1.2
+-------------
+
+*(Released Fri, 7 May 2010)*
+
+Another release with a long development cycle, during which many
+different features were added.
+
+Significant features
+~~~~~~~~~~~~~~~~~~~~
+
+The KVM hypervisor now can run the individual instances as non-root, to
+reduce the impact of a VM being hijacked due to bugs in the
+hypervisor. It is possible to run all instances as a single (non-root)
+user, to manually specify a user for each instance, or to dynamically
+allocate a user out of a cluster-wide pool to each instance, with the
+guarantee that no two instances will run under the same user ID on any
+given node.
+
+An experimental RAPI client library, that can be used standalone
+(without the other Ganeti libraries), is provided in the source tree as
+``lib/rapi/client.py``. Note this client might change its interface in
+the future, as we iterate on its capabilities.
+
+A new command, ``gnt-cluster renew-crypto`` has been added to easily
+replace the cluster's certificates and crypto keys. This might help in
+case they have been compromised, or have simply expired.
+
+A new disk option for instance creation has been added that allows one
+to "adopt" currently existing logical volumes, with data
+preservation. This should allow easier migration to Ganeti from
+unmanaged (or managed via other software) instances.
+
+Another disk improvement is the possibility to convert between redundant
+(DRBD) and plain (LVM) disk configuration for an instance. This should
+allow better scalability (starting with one node and growing the
+cluster, or shrinking a two-node cluster to one node).
+
+A new feature that could help with automated node failovers has been
+implemented: if a node sees itself as offline (by querying the master
+candidates), it will try to shutdown (hard) all instances and any active
+DRBD devices. This reduces the risk of duplicate instances if an
+external script automatically failovers the instances on such nodes. To
+enable this, the cluster parameter ``maintain_node_health`` should be
+enabled; in the future this option (per the name) will enable other
+automatic maintenance features.
+
+Instance export/import now will reuse the original instance
+specifications for all parameters; that means exporting an instance,
+deleting it and the importing it back should give an almost identical
+instance. Note that the default import behaviour has changed from
+before, where it created only one NIC; now it recreates the original
+number of NICs.
+
+Cluster verify has added a few new checks: SSL certificates validity,
+/etc/hosts consistency across the cluster, etc.
+
+Other changes
+~~~~~~~~~~~~~
+
+As usual, many internal changes were done, documentation fixes,
+etc. Among others:
+
+- Fixed cluster initialization with disabled cluster storage (regression
+ introduced in 2.1.1)
+- File-based storage supports growing the disks
+- Fixed behaviour of node role changes
+- Fixed cluster verify for some corner cases, plus a general rewrite of
+ cluster verify to allow future extension with more checks
+- Fixed log spamming by watcher and node daemon (regression introduced
+ in 2.1.1)
+- Fixed possible validation issues when changing the list of enabled
+ hypervisors
+- Fixed cleanup of /etc/hosts during node removal
+- Fixed RAPI response for invalid methods
+- Fixed bug with hashed passwords in ``ganeti-rapi`` daemon
+- Multiple small improvements to the KVM hypervisor (VNC usage, booting
+ from ide disks, etc.)
+- Allow OS changes without re-installation (to record a changed OS
+ outside of Ganeti, or to allow OS renames)
+- Allow instance creation without OS installation (useful for example if
+ the OS will be installed manually, or restored from a backup not in
+ Ganeti format)
+- Implemented option to make cluster ``copyfile`` use the replication
+ network
+- Added list of enabled hypervisors to ssconf (possibly useful for
+ external scripts)
+- Added a new tool (``tools/cfgupgrade12``) that allows upgrading from
+ 1.2 clusters
+- A partial form of node re-IP is possible via node readd, which now
+ allows changed node primary IP
+- Command line utilities now show an informational message if the job is
+ waiting for a lock
+- The logs of the master daemon now show the PID/UID/GID of the
+ connected client
+
+
+Version 2.1.1
+-------------
+
+*(Released Fri, 12 Mar 2010)*
+
+During the 2.1.0 long release candidate cycle, a lot of improvements and
+changes have accumulated with were released later as 2.1.1.
+
+Major changes
+~~~~~~~~~~~~~
+
+The node evacuate command (``gnt-node evacuate``) was significantly
+rewritten, and as such the IAllocator protocol was changed - a new
+request type has been added. This unfortunate change during a stable
+series is designed to improve performance of node evacuations; on
+clusters with more than about five nodes and which are well-balanced,
+evacuation should proceed in parallel for all instances of the node
+being evacuated. As such, any existing IAllocator scripts need to be
+updated, otherwise the above command will fail due to the unknown
+request. The provided "dumb" allocator has not been updated; but the
+ganeti-htools package supports the new protocol since version 0.2.4.
+
+Another important change is increased validation of node and instance
+names. This might create problems in special cases, if invalid host
+names are being used.
+
+Also, a new layer of hypervisor parameters has been added, that sits at
+OS level between the cluster defaults and the instance ones. This allows
+customisation of virtualization parameters depending on the installed
+OS. For example instances with OS 'X' may have a different KVM kernel
+(or any other parameter) than the cluster defaults. This is intended to
+help managing a multiple OSes on the same cluster, without manual
+modification of each instance's parameters.
+
+A tool for merging clusters, ``cluster-merge``, has been added in the
+tools sub-directory.
+
+Bug fixes
+~~~~~~~~~
+
+- Improved the int/float conversions that should make the code more
+ robust in face of errors from the node daemons
+- Fixed the remove node code in case of internal configuration errors
+- Fixed the node daemon behaviour in face of inconsistent queue
+ directory (e.g. read-only file-system where we can't open the files
+ read-write, etc.)
+- Fixed the behaviour of gnt-node modify for master candidate demotion;
+ now it either aborts cleanly or, if given the new “auto_promote”
+ parameter, will automatically promote other nodes as needed
+- Fixed compatibility with (unreleased yet) Python 2.6.5 that would
+ completely prevent Ganeti from working
+- Fixed bug for instance export when not all disks were successfully
+ exported
+- Fixed behaviour of node add when the new node is slow in starting up
+ the node daemon
+- Fixed handling of signals in the LUXI client, which should improve
+ behaviour of command-line scripts
+- Added checks for invalid node/instance names in the configuration (now
+ flagged during cluster verify)
+- Fixed watcher behaviour for disk activation errors
+- Fixed two potentially endless loops in http library, which led to the
+ RAPI daemon hanging and consuming 100% CPU in some cases
+- Fixed bug in RAPI daemon related to hashed passwords
+- Fixed bug for unintended qemu-level bridging of multi-NIC KVM
+ instances
+- Enhanced compatibility with non-Debian OSes, but not using absolute
+ path in some commands and allowing customisation of the ssh
+ configuration directory
+- Fixed possible future issue with new Python versions by abiding to the
+ proper use of ``__slots__`` attribute on classes
+- Added checks that should prevent directory traversal attacks
+- Many documentation fixes based on feedback from users
+
+New features
+~~~~~~~~~~~~
+
+- Added an “early_release” more for instance replace disks and node
+ evacuate, where we release locks earlier and thus allow higher
+ parallelism within the cluster
+- Added watcher hooks, intended to allow the watcher to restart other
+ daemons (e.g. from the ganeti-nbma project), but they can be used of
+ course for any other purpose
+- Added a compile-time disable for DRBD barriers, to increase
+ performance if the administrator trusts the power supply or the
+ storage system to not lose writes
+- Added the option of using syslog for logging instead of, or in
+ addition to, Ganeti's own log files
+- Removed boot restriction for paravirtual NICs for KVM, recent versions
+ can indeed boot from a paravirtual NIC
+- Added a generic debug level for many operations; while this is not
+ used widely yet, it allows one to pass the debug value all the way to
+ the OS scripts
+- Enhanced the hooks environment for instance moves (failovers,
+ migrations) where the primary/secondary nodes changed during the
+ operation, by adding {NEW,OLD}_{PRIMARY,SECONDARY} vars
+- Enhanced data validations for many user-supplied values; one important
+ item is the restrictions imposed on instance and node names, which
+ might reject some (invalid) host names
+- Add a configure-time option to disable file-based storage, if it's not
+ needed; this allows greater security separation between the master
+ node and the other nodes from the point of view of the inter-node RPC
+ protocol
+- Added user notification in interactive tools if job is waiting in the
+ job queue or trying to acquire locks
+- Added log messages when a job is waiting for locks
+- Added filtering by node tags in instance operations which admit
+ multiple instances (start, stop, reboot, reinstall)
+- Added a new tool for cluster mergers, ``cluster-merge``
+- Parameters from command line which are of the form ``a=b,c=d`` can now
+ use backslash escapes to pass in values which contain commas,
+ e.g. ``a=b\\c,d=e`` where the 'a' parameter would get the value
+ ``b,c``
+- For KVM, the instance name is the first parameter passed to KVM, so
+ that it's more visible in the process list
+
+
+Version 2.1.0
+-------------
+
+*(Released Tue, 2 Mar 2010)*
+
+Ganeti 2.1 brings many improvements with it. Major changes:
+
+- Added infrastructure to ease automated disk repairs
+- Added new daemon to export configuration data in a cheaper way than
+ using the remote API
+- Instance NICs can now be routed instead of being associated with a
+ networking bridge
+- Improved job locking logic to reduce impact of jobs acquiring multiple
+ locks waiting for other long-running jobs
+
+In-depth implementation details can be found in the Ganeti 2.1 design
+document.
+
+Details
+~~~~~~~
+
+- Added chroot hypervisor
+- Added more options to xen-hvm hypervisor (``kernel_path`` and
+ ``device_model``)
+- Added more options to xen-pvm hypervisor (``use_bootloader``,
+ ``bootloader_path`` and ``bootloader_args``)
+- Added the ``use_localtime`` option for the xen-hvm and kvm
+ hypervisors, and the default value for this has changed to false (in
+ 2.0 xen-hvm always enabled it)
+- Added luxi call to submit multiple jobs in one go
+- Added cluster initialization option to not modify ``/etc/hosts``
+ file on nodes
+- Added network interface parameters
+- Added dry run mode to some LUs
+- Added RAPI resources:
+
+ - ``/2/instances/[instance_name]/info``
+ - ``/2/instances/[instance_name]/replace-disks``
+ - ``/2/nodes/[node_name]/evacuate``
+ - ``/2/nodes/[node_name]/migrate``
+ - ``/2/nodes/[node_name]/role``
+ - ``/2/nodes/[node_name]/storage``
+ - ``/2/nodes/[node_name]/storage/modify``
+ - ``/2/nodes/[node_name]/storage/repair``
+
+- Added OpCodes to evacuate or migrate all instances on a node
+- Added new command to list storage elements on nodes (``gnt-node
+ list-storage``) and modify them (``gnt-node modify-storage``)
+- Added new ssconf files with master candidate IP address
+ (``ssconf_master_candidates_ips``), node primary IP address
+ (``ssconf_node_primary_ips``) and node secondary IP address
+ (``ssconf_node_secondary_ips``)
+- Added ``ganeti-confd`` and a client library to query the Ganeti
+ configuration via UDP
+- Added ability to run hooks after cluster initialization and before
+ cluster destruction
+- Added automatic mode for disk replace (``gnt-instance replace-disks
+ --auto``)
+- Added ``gnt-instance recreate-disks`` to re-create (empty) disks
+ after catastrophic data-loss
+- Added ``gnt-node repair-storage`` command to repair damaged LVM volume
+ groups
+- Added ``gnt-instance move`` command to move instances
+- Added ``gnt-cluster watcher`` command to control watcher
+- Added ``gnt-node powercycle`` command to powercycle nodes
+- Added new job status field ``lock_status``
+- Added parseable error codes to cluster verification (``gnt-cluster
+ verify --error-codes``) and made output less verbose (use
+ ``--verbose`` to restore previous behaviour)
+- Added UUIDs to the main config entities (cluster, nodes, instances)
+- Added support for OS variants
+- Added support for hashed passwords in the Ganeti remote API users file
+ (``rapi_users``)
+- Added option to specify maximum timeout on instance shutdown
+- Added ``--no-ssh-init`` option to ``gnt-cluster init``
+- Added new helper script to start and stop Ganeti daemons
+ (``daemon-util``), with the intent to reduce the work necessary to
+ adjust Ganeti for non-Debian distributions and to start/stop daemons
+ from one place
+- Added more unittests
+- Fixed critical bug in ganeti-masterd startup
+- Removed the configure-time ``kvm-migration-port`` parameter, this is
+ now customisable at the cluster level for both the KVM and Xen
+ hypervisors using the new ``migration_port`` parameter
+- Pass ``INSTANCE_REINSTALL`` variable to OS installation script when
+ reinstalling an instance
+- Allowed ``@`` in tag names
+- Migrated to Sphinx (http://sphinx.pocoo.org/) for documentation
+- Many documentation updates
+- Distribute hypervisor files on ``gnt-cluster redist-conf``
+- ``gnt-instance reinstall`` can now reinstall multiple instances
+- Updated many command line parameters
+- Introduced new OS API version 15
+- No longer support a default hypervisor
+- Treat virtual LVs as inexistent
+- Improved job locking logic to reduce lock contention
+- Match instance and node names case insensitively
+- Reimplemented bash completion script to be more complete
+- Improved burnin
+
+