Queries Endpoint objects (using plugins in parallel) and sends the found entities to consumers. More...

#include <arc/compute/EntityRetriever.h>

Inheritance diagram for Arc::EntityRetriever< T >:

Data Structures
class	Common

class	Result

class	ThreadArg

Public Member Functions
	EntityRetriever (const UserConfig &uc, const EndpointQueryOptions< T > &options=EndpointQueryOptions< T >())
	Needs the credentials of the user and can have some options. More...

void	wait () const

bool	isDone () const

void	addConsumer (EntityConsumer< T > &addConsumer_consumer)

void	removeConsumer (const EntityConsumer< T > &removeConsumer_consumer)

EndpointQueryingStatus	getStatusOfEndpoint (const Endpoint &endpoint) const
	Get the status of the query process of a given Endpoint. More...

EndpointStatusMap	getAllStatuses () const
	Get status of all the queried Endpoint objects. More...

bool	setStatusOfEndpoint (const Endpoint &endpoint, const EndpointQueryingStatus &status, bool overwrite=true)
	Set the status of the query process of a given Endpoint. More...

void	getServicesWithStatus (const EndpointQueryingStatus &status, std::set< std::string > &result)
	Insert into results the endpoint.ServiceName() of each endpoint with the given status. More...

void	clearEndpointStatuses ()
	Clear statuses of registered endpoints. More...

bool	removeEndpoint (const Endpoint &e)
	Remove a particular registered endpoint. More...

virtual void	addEntity (const T &entity)

virtual void	addEndpoint (const Endpoint &endpoint)
	Starts querying an Endpoint. More...

void	needAllResults (bool all_results=true)
	Sets if all wait for all queries. More...

Protected Member Functions
void	checkSuspendedAndStart (const Endpoint &e)

Static Protected Member Functions
static void	queryEndpoint (void *arg_)

Protected Attributes
ThreadedPointer< Common >	common

Result	result

EndpointStatusMap	statuses

const UserConfig &	uc

std::list< EntityConsumer< T > *>	consumers

const EndpointQueryOptions< T >	options

SimpleCondition	consumerLock

SimpleCondition	statusLock

std::map< std::string, std::string >	interfacePluginMap

bool	need_all_results

Static Protected Attributes
static Logger	logger

Detailed Description

template<typename T>
class Arc::EntityRetriever< T >

Queries Endpoint objects (using plugins in parallel) and sends the found entities to consumers.

The EntityRetriever is a template class which queries Endpoint objects and returns entities of the template type T. The query is done by plugins (capable of retrieving type T objects from Endpoint objects), and the results are sent to the registered EntityConsumer objects (capable of consuming type T objects).

When an Endpoint is added to the EntityRetriever, a new thread is started which queries the given Endpoint. Each plugin is capable of querying Endpoint objects with given interfaces (which is indicated with the InterfaceName attribute of the Endpoint). If the Endpoint has the InterfaceName specified, then the plugin capable of querying that interface will be selected. If the InterfaceName of the Endpoint is not specified, all the available plugins will be considered. If there is a preferred list of interfaces, then first the plugins supporting those interfaces will be tried, and if there are no preferred interfaces, or the preferred ones did not give any result, then all the plugins will be tried. All this happens parallel in separate threads. Currently there are three instance classes:

the ServiceEndpointRetriever queries service registries and returns new Endpoint objects
the TargetInformationRetriever queries computing elements and returns reource information in form of ComputingServiceType objects
the JobListRetriever queries computing elements and returns information about the jobs at the computing element in form of Job objects

To start querying, a new EntityRetriever needs to be created with the user's credentials in the UserConfig object, then one or more consumers needs to be added with the addConsumer method (e.g. an EntityContainer of the given T type), then the Endpoints need to be added one by one with the addEndpoint method. Then the wait method can be called to wait for all the results to arrive, after which we can be sure that all the retrieved entities are passed to the registered consumer objects. If we registered an EntityContainer, then we can get all the results from the container, using it as a standard list.

It is possible to specify options in the constructor, which in case of the TargetInformationRetriever and the JobListRetriever classes is an EndpointQueryOptions object containing a list of preferred InterfaceNames. When an Endpoint has not InterfaceName specified, these preferred InterfaceNames will be tried first. The ServiceEndpointRetriever has different options though: the EndpointQueryOptions<Endpoint> object does not contain a preferred list of InterfaceNames. It has a flag for recursivity instead and string lists for filtering services by capability and rejecting them by URL.

See also: ComputingServiceRetriever which combines the ServiceEndpointRetriever and the TargetInformationRetriever to query both the service registries and the computing elements

ServiceEndpointRetriever example:

Arc::UserConfig uc;
// create the retriever with no options
Arc::ServiceEndpointRetriever retriever(uc);
// create a container which will store the results
Arc::EntityContainer<Arc::Endpoint> container;
// add the container to the retriever
retriever.addConsumer(container);
// create an endpoint which will be queried
Arc::Endpoint registry("test.nordugrid.org", Arc::Endpoint::REGISTRY);
// start querying the endpoint
retriever.addEndpoint(registry);
// wait for the querying process to finish
retriever.wait();
// get the status of the query
Arc::EndpointQueryingStatus status = retriever.getStatusOfEndpoint(registry);

After wait returns, container contains all the services found in the registry "test.nordugrid.org".

TargetInformationRetriever example:

Arc::UserConfig uc;
// create the retriever with no options
Arc::TargetInformationRetriever retriever(uc);
// create a container which will store the results
Arc::EntityContainer<Arc::ComputingServiceType> container;
// add the container to the retriever
retriever.addConsumer(container);
// create an endpoint which will be queried
Arc::Endpoint ce("test.nordugrid.org", Arc::Endpoint::COMPUTINGINFO);
// start querying the endpoint
retriever.addEndpoint(ce);
// wait for the querying process to finish
retriever.wait();
// get the status of the query
Arc::EndpointQueryingStatus status = retriever.getStatusOfEndpoint(ce);

After wait returns, container contains the ComputingServiceType object which has the full GLUE2 information about the computing element.

Constructor & Destructor Documentation

◆ EntityRetriever()

template<typename T>

Arc::EntityRetriever< T >::EntityRetriever	(	const UserConfig &	uc,
		const EndpointQueryOptions< T > &	options = `EndpointQueryOptions< T >()`
	)

Needs the credentials of the user and can have some options.

Creating the EntityRetriever does not start any querying yet.

Parameters

uc	UserConfig with the user's credentials
options	contain type T specific querying options

Member Function Documentation

◆ addConsumer()

template<typename T>

void Arc::EntityRetriever< T >::addConsumer ( EntityConsumer< T > & addConsumer_consumer )

inline

Register a new consumer which will receive results from now on.

Parameters

[in] addConsumer_consumer is a consumer object capable of consuming type T objects

◆ addEndpoint()

template<typename T>

virtual void Arc::EntityRetriever< T >::addEndpoint ( const Endpoint & endpoint )

virtual

Starts querying an Endpoint.

This method is used to start querying an Endpoint. It starts the query process in a separate thread, and returns immediately.

Parameters

[in] endpoint is the Endpoint to query

◆ addEntity()

template<typename T>

virtual void Arc::EntityRetriever< T >::addEntity ( const T & entity )

virtual

This method should only be used by the plugins when they return their results. This will send the results to all the registered consumers.

In the case of the ServiceEndpointRetriever, the retrieved entities are actually Endpoint objects, and the ServiceEndpointRetriever does more work here depending on the options set in EndpointQueryOptions<Endpoint>:

if the URL of a retrieved Endpoint is on the rejected list, the Endpoint is not sent to the consumers
if recursivity is turned on, and the retrieved Endpoint is a service registry, then it is sent to the addEntity method for querying
if the retrieved Endpoint does not have at least one of the capabilities provided in the capability filter, then the Endpoint is not sent to the consumers
Parameters

[in] entity is the type T object retrieved from the endpoints

Implements Arc::EntityConsumer< T >.

◆ clearEndpointStatuses()

template<typename T>

void Arc::EntityRetriever< T >::clearEndpointStatuses ( )

inline

Clear statuses of registered endpoints.

The status map of registered endpoints will be cleared when calling this method. That can be useful if an already registered endpoint need to be queried again.

◆ getAllStatuses()

template<typename T>

EndpointStatusMap Arc::EntityRetriever< T >::getAllStatuses ( ) const

inline

Get status of all the queried Endpoint objects.

This method returns a copy of the internal status map, and thus is only a snapshot. If you want the final status map, make sure to invoke the EntityRetriever::wait method before this one.

Returns: a map with Endpoint objects as keys and status objects as values.

◆ getServicesWithStatus()

template<typename T>

void Arc::EntityRetriever< T >::getServicesWithStatus	(	const EndpointQueryingStatus &	status,
		std::set< std::string > &	result
	)

Insert into results the endpoint.ServiceName() of each endpoint with the given status.

Parameters

[in]	status	is the status of the desired endpoints
[in,out]	result	is a set into which the matching endpoint service names are inserted

◆ getStatusOfEndpoint()

template<typename T>

EndpointQueryingStatus Arc::EntityRetriever< T >::getStatusOfEndpoint ( const Endpoint & endpoint ) const

Get the status of the query process of a given Endpoint.

Parameters

[in] endpoint is the Endpoint whose status we want to know.

Returns: an EndpointQueryingStatus object containing the status of the query

◆ isDone()

template<typename T>

bool Arc::EntityRetriever< T >::isDone ( ) const

inline

Check if the query is finished.

Returns: true if the query is finished, all the results were delivered to the consumers.

◆ needAllResults()

template<typename T>

void Arc::EntityRetriever< T >::needAllResults ( bool all_results = true )

inline

Sets if all wait for all queries.

This method specifies if whole query must wait for all individual queries to same endpoint to fnish. By default it waits for first successful one. But in some cases it may be needed to obtain results from all available interfaces because they may be different.

◆ removeConsumer()

template<typename T>

void Arc::EntityRetriever< T >::removeConsumer ( const EntityConsumer< T > & removeConsumer_consumer )

Remove a previously registered consumer

Parameters

[in] removeConsumer_consumer is the consumer object

◆ removeEndpoint()

template<typename T>

bool Arc::EntityRetriever< T >::removeEndpoint ( const Endpoint & e )

inline

Remove a particular registered endpoint.

The specified endpoint will be removed from the status map of registered endpoints.

Parameters

e	endpoint to remove from status map.

Returns: true is returned if endpoint is found in the map, otherwise false is returned.

◆ setStatusOfEndpoint()

template<typename T>

bool Arc::EntityRetriever< T >::setStatusOfEndpoint	(	const Endpoint &	endpoint,
		const EndpointQueryingStatus &	status,
		bool	overwrite = `true`
	)

Set the status of the query process of a given Endpoint.

This method should only be used by the plugins when they finished querying an Endpoint.

Parameters

[in]	endpoint	is the Endpoint whose status we want to set
[in]	status	is the EndpointQueryStatus object containing the status
[in]	overwrite	indicates if a previous status should be overwritten, if not, then in case of an existing status the method returns false

Returns: true if the new status was set, false if it was not set (e.g. because overwrite was false, and the status was already set previously)

◆ wait()

template<typename T>

void Arc::EntityRetriever< T >::wait ( void ) const

inline

This method blocks until all the results arrive.

The documentation for this class was generated from the following file:

EntityRetriever.h

Data Structures

Public Member Functions

Protected Member Functions

Static Protected Member Functions

Protected Attributes

Static Protected Attributes

Detailed Description

template<typename T> class Arc::EntityRetriever< T >

Constructor & Destructor Documentation

◆ EntityRetriever()

Member Function Documentation

◆ addConsumer()

◆ addEndpoint()

◆ addEntity()

◆ clearEndpointStatuses()

◆ getAllStatuses()

◆ getServicesWithStatus()

◆ getStatusOfEndpoint()

◆ isDone()

◆ needAllResults()

◆ removeConsumer()

◆ removeEndpoint()

◆ setStatusOfEndpoint()

◆ wait()

template<typename T>
class Arc::EntityRetriever< T >