http.client: Disable SSL session ID cache
This patch disables the SSL session ID cache for all cURL operations.This is needed because http.HttpBase's PyOpenSSL implementation does notcurrently set a context using SSL_set_session_id_context(3SSL), cURLtries to re-use the session ID and, according to...
http.auth: Fix docstring error
This was missing from commit 2287b920.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'stable-2.2'
http.auth.ReadPasswordFile: Don't read file directly
Reading the file before this function allows for better errorreporting.
Set list of trusted SSL CAs for client to verify
As per SSL_CTX_set_client_CA_list(3SSL), set the list of acceptable CAsadvertised to SSL clients to include the server's own certificate. Thisevidently fixes the pycurl/gnutls RPC client.
During the TLS Handshake, when client verification is requested, the...
Merge branch 'devel-2.2'
Fix pylint warning in http/__init__.py
My bad for not seeing this before:R0201:614:HttpBase.GetSslCiphers: Method could be a function
Allow SSL ciphers to be overridden in HTTP server
Users of this class, such as the RAPI server, might want to override or adjustthe default SSL cipher defined in a constant.
Make family argument in FormatAddress optional
By doing this we delegate the task of finding the correct address familyto the FormatAddress method.
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
rapi.client, http.client: Format url correctly when using IPv6
This patch moves the FormatAddress helper function from daemon.py tonetutils.py. This enables its use in http.client as well as inrapi.client. Furthermore this adds functionality to format IPv6...
Support IPv6 in lib/http/server.py
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Convert RPC client to PycURL
Instead of using our custom HTTP client, using PycURL's multiinterface allows us to get rid of the HTTP client threadpool.The majority of the code is still in the ganeti.http.clientmodule.
A simple per-thread HTTP client pool gives cURL a chance to...
Disallow DES for SSL connections
Older OpenSSL versions include DES-CBC3-* ciphers when specifying theHIGH group of ciphers. Removing potentially weak ciphers from the listof allowed ciphers ensures only strong ciphers are considered for SSLconnections....
http client: support per-request read timeout
Currently, the read timeout is hardcoded in theHttpClientRequestExecutor class. The patch changes the timeout so thatit's a per-request property, and makes the rpc.Client class pass oneexplicitly in. Furthermore, we modify the rpc.RpcRunner class to support...
Move hash functions to the compat module
Since the hash functions' changed their module name between python 2.4and 2.6, and we have to do an try/import/except trick, we'll do it justonce, for both hash functions, and in compat.py. This also fixes a use...
Merge branch 'devel-2.1'
move http.WaitForSocketCondition to utils
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
WaitForSocketCondition: rename, handle EINTR
- Rename WaitForSocketCondition to SingleWaitForFdCondition - Avoid potentially infinite loop, if we continue to get interrupted - Handle eintr correctly - Avoid the poller try/finally, as the poller object gets destroyed...
Merge remote branch 'origin/devel-2.1'
http.auth: Disable pylint warnings
http.server: Improve request logging in debug mode
Provide unittests for http.auth
To simplify writing unittests, one data structure class in http.server isalso changed. According to the coverage utility, this provides 95%coverage.
http.auth: Fix bug with checking hashed passwords
When username and password were sent for a resource not requiringauthentication, it wouldn't be accepted if the user in question had ahashed password. The reason was that the function GetAuthRealm used to...
Merge remote branch 'origin/devel-2.0' into devel-2.1
Conflicts: NEWS: Trivial configure.ac: Trivial...
Fix two potentially endless loops in http library
The first can be problematic if poll(2) returns POLLHUP|POLLERR on asocket. Before it would be only be respected for SOCKOP_RECV, but sincethey can also occur on other socket operations, esp. in combination with...
Reset tempfile module after fork where useful
Merge branch 'devel-2.0' into devel-2.1
Conflicts: lib/backend.py - trivial merge...
Ensure all int/float conversions are handled right
int()/float() can raise either ValueError (in case of int("a")), orTypeError (in case of int(None)). We had many bugs over time due tothis, and a recent one was just diagnosed, so we go over the codebase...
Remove http.HttpJsonConverter
With the move of the content-type handling to the various users of the HTTPlayer, this class isn't really useful anymore.
http.server: No longer enfore JSON encoding for body
The HTTP layer shouldn't care about the contents of the request data orresponses. This requires further changes in the RAPI code to handle clientrequests correctly.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
http.server: Refuse HTTP/1.1 request without Host header
http: Add two new exceptions, one constant
These will be useful in the future in case we don't enfore JSON encodinganymore in the http.server module. The HTTP 1.1 RFC recommends error 415(Unsupported Media Type) to be returned in case the client requests an...
Factorize LUXI parsing and handling code
Also fix a typo in http/__init__.py and add unittestsfor the LUXI parsing and formatting functions.
Improve logging for workerpool tasks by providing repr
Before it would log something like “starting task(<ganeti.http.client._HttpClientPendingRequest object at 0x2aaaad176790>,)”,which isn't really useful for debugging. Now it'll log “[…]<ganeti.http.client._HttpClientPendingRequest...
workerpool: Make worker ID alphanumeric
Having a proper name instead of just a number makes debuggingeasier.
Merge remote branch 'devel-2.1'
Fix unused imports or add silences where needed
In some cases pylint doesn't parse the import correctly, so we addsilences; but there are also many cases of unused imports, which wesimply remove.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
Further pylint disables, mostly for Unused args
Many of our functions have to follow a given API, and thus we have tokeep a given signature, but pylint doesn't understand this. Therefore,we silence this warning.
The patch does a few other cleanups.
Signed-off-by: Iustin Pop <iustin@google.com>...
Convert to static methods (where appropriate)
Many methods are simple pure functions, and not depending on the objectstate. We convert these to staticmethods.
Add targeted pylint disables
This patch should have only:
- pylint disables- docstring changes- whitespace changes
Remove many 'Unused variable' warnings
Note there are some cases left which need extra cleanup.
Add targetted pylint disables
This patch adds targeted pylint disables, where it makes sense (eitherdue to limitations in pylint or due to historical usage), and also a fewblanket ones in rapi where all the names are… “different”.
A few style updates
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add check for OpenSSL entropy status
By checking for this explicitly, the errors (SSLEAY_RAND_BYTES, “PRNGnot seeded”) will happen in the start-up phase of the daemon and notonly when executing remote procedure calls.
Fix pylint 'E' (error) codes
This patch adds some silences and tweaks the code slightly so that“pylint --rcfile pylintrc -e ganeti” doesn't give any errors.
The biggest change is in jqueue.py, the move of _RequireOpenQueue out ofthe JobQueue class. Since that is actually a function and not a method...
Epydoc fixes
http.auth: Add new function to verify passwords
This new function supports two schemes for passwords:- Old-style cleartext passwords- Hashed passwords according to RFC2617 (H(A1))
Schemes are differentiated by their prefix, a concept alsoused in OpenLDAP. Cleartext passwords can no longer start...
ganeti-noded: Close listening socket in child
Convert the http server/mainloop to asyncore
We can avoid most of the Mainloop.Run() code if we use asyncorefor delivering I/O events, and just concentrate on what's missing inasyncore: singnal handling and timers. This way confd can be ported touse Mainloop as well....
Fix pylint warnings
Fix some typos
Fix HTTP server library handling of credentials
Currently the http library only checks credentials when authenticationis required. This means that any credentials are accepted on the rootresource, for example, which makes problems hard to diagnose - the...
Fix _NOQUOTE regexp
Allow expressions longer than one character to match.
Reviewed-by: imsnah
Fix some epydoc style issues
99% of the epydoc return tags are "@return:", but each of the modified fileshad one "@returns:" line. We fix this for consistency.
RAPI: format error messages as JSON
This patch changes the format of the HTTP error messages from text/html, whichis hard to parse from RAPI clients, to JSON which can be automatically parsed.
The error message is an object, which contains always three keys:...
Make RAPI return 502/504 errors for luxi errors
This changes the RAPI error codes for luxi errors; a timeout error isnow reported properly as 504, while any other luxi error is reported as502.
It would be good to convert even more errors into proper return codes in...
rapi: fix authentication and queries
For queries, we don't want to require authentication. We fix this by adding anoverride GetAuthRealm in the rapi daemon.
We also fix a method name.
Some docstring updates
This patch rewraps some comments to shorter lengths, changesdouble-quotes to single-quotes inside triple-quoted docstrings forbetter editor handling.
It also fixes some epydoc errors, namely invalid crossreferences (aftermethod rename), documentation for inexistent (removed) parameters, etc....
ganeti-noded: reduce log noise
The source port/addr is currently logged three times for eachconnection, and this is unnecessary. We change two log entries to debug,since they are useful for precise timing, and we keep only one at INFOlevel.
Fix some pylint-detected issues
Two bad indentation cases and a missing variable.
ganeti.http: Function to read password file
Lines in the password file are of the following format:
<username> <password> [options]
Fields are separated by whitespace. Username and password aremandatory, options are optional and separated by comma (",")....
ganeti.http: Add support for private data in HTTP requests
Reviewed-by: amishchenko
ganeti.http: Add support for basic HTTP authentication
As per RFC2617.
ganeti.http: Prepare authentication for HTTP server
The authentication class will override PreHandleRequest.
ganeti.http: Don't pass poller object around
They're cheap to instantiate and doing this changes makes the codea bit simpler.
Reviewed-by: ultrotter
Rename http.HttpInternalError to HttpInternalServerError
All other exceptions are named after the error name in RFC2616 (HTTP/1.1).
ganeti.http: Add more constants and errors
ganeti.http: Ignore ENOTCONN when shutting down the connection
Implement support for additional headers with HTTP errors
Add simple unittests for ganeti.http
More complex unittests will need some refactoring in the HTTP code.
ganeti.http: Add three TODOs for improvements
Reviewed-by: iustinp
ganeti.http: Explicitly initiate handshake
Otherwise it would be done on the first read/write operation, makingerror handling more difficult (such as EOF during handshake).
ganeti.http: Implement handshake socket operation
ganeti.http: Handle SSL_ERROR_ZERO_RETURN
Also add a comment next to the place where the SSL connection is shutdown.
cleanup: http server, line too long
cleanup: http client, line too long
ganeti.http: Rename HttpBase._using_ssl to HttpBase.using_ssl
It'll be queried from other classes.
ganeti.http: Rename HttpSocketBase to HttpBase
It's more appropriate.
Fix epydoc format warnings
This patch should fix all outstanding epydoc parsing errors; as such, weswitch epydoc into verbose mode so that any new errors will be visible.
ganeti.http: Docstring updates
ganeti.http: Remove _HttpClientError
This is a leftover from old code.
ganeti.http.server: Increase connection backlog to 1024
This solves a problem with many concurrent requests. By default, 1024is the maximum backlog on Linux kernels. We limit the number of clientsthrough MAX_CHILDREN, too. The idea of just increasing the backlog is...
http: use slicing instead of string modification
The combination of the current buffer splitting method and (4KB) buffersize is very inefficient when writing big amounts of data. Just walkingover a 16 megabyte string using a 4K buffer takes (on a random computer)...
ganeti.http: Add constant for DELETE
Remove old HTTP code
ganeti.http: Split HTTP server and client into separate files
This includes a large rewrite of the HTTP server code. The handling ofOpenSSL errors had some problems that were hard to fix with itsstructure. When preparing all of this, I realized that actually HTTP...
Rename all HTTP classes to camel case
It should be consistent.
ganeti.http: Remove underline from two classes
This is a preparation step for splitting the HTTP client and server codeinto two separate modules.
Move HTTP code to subpackage