db

class db.Proxy(config)

This is the frontend class for interaction with the database. It supports either a local TinyDB database or a remote MongoDB database.

Most methods tend to favor interacting with the remote database if present, and fallback to a local database if not. Queries are unified using the TinyDB.Query class, which is translated to a native MongoDB query when interacting with the remote database. This has limitations since MongoDB queries are more expressive but we can always use directly the Proxy.rdb instance to avoid this limitation.

set_tag(tag=None)

Sets the global “tag” added to all queries performed by the Proxy to filter the database by document.tag values.

insert_multiple(docs)

Inserts multiple documents in the local database only.

contains(q=None, **kargs)

Returns True if the provided query ‘q’ matches a document (on the filtered by self.tag set of document from remote if present otherwise local.)

search(q=None, **kargs)

Returns the list of documents matching the provided query ‘q’ (on the filtered by self.tag set of document from remote if present otherwise local.)

get(q=None, **kargs)

Returns the first document matching the provided query ‘q’ (on the filtered by self.tag set of document from remote if present otherwise local.)

cleanup_local()

Removes duplicates from the local database only. (Documents are considered duplicates if their “id” and “val” attributes are equal.)

cleanup()

Wrapper for remote cleanup method.

find_matching_types(Locs, req=None, psize=0)

Wrapper for find_matching_types method.

close()

Close the local database only.

class db.MongoDB(url, auth=None, verify=True)

This class implements the interface with a mongodb server.

The database used is named “ccrawl”, and documents are all stored in the “nodes” collection. Collections struct_ptr32 and struct_ptr64 are used to precompute structures’ offsets assuming respective pointer size of 32 bits and 64 bits.

insert_multiple(docs)

Calls insert_many on the nodes collection.

contains(q, **kargs)

Calls find on the nodes collection for the given query. with limit set to 1 and returns True if the list is non empty.

search(q, **kargs)

Calls find on the nodes collection for the given query.

get(q, **kargs)

Calls find_one on the nodes collection for the given query.

cleanup(proxy)

Removes duplicates and their precomputed data in structs_ptr32/64 collections and re-index the database by “id” and “val” fields.

cleanup_structs(**kargs)

Remove all entries from struct_ptr32/64 collections that don’t have a matching _id in the nodes collection.

cleanup_selected(**kargs)

Remove all entries that matches passed key=value arguments (from all collections.)

update_structs(proxydb, req=None)

Update the struct_32/64 collections for the given request filtered documents.

remove_duplicates(**kargs)

Remove duplicates from the nodes collection.

find_matching_types(Locs, req=None, psize=0)

For a given dictionnary of “locations” where the key is an arbitrary name (a stack variable name, etc) and the value is a list of (offset,size) tuples that describe how this local stack variable is accessed within a function, try to find any matching structured type in the database and updates the dict value by adding these typenames.

find_calls_to(ref, D=None)

Find cFunc documents that calls a given ref name. This works only for functions that have been collected with the –all option.