/
opt
/
gsutil
/
gslib
/
__pycache__
/
up file
home
a +(Wg�6 � @ s� d Z ddlmZ ddlmZ ddlmZ ddlmZ ddlZddlZddlm Z ddl m Z G d d � d e�ZG d d � d e�ZG d d� de�ZG dd� de�ZG dd� de�ZG dd� de�ZG dd� de�ZG dd� de�ZG dd� de�ZdS )zAThread message classes. Messages are added to the status queue. � )�absolute_import)�print_function)�division)�unicode_literalsN)�Error)�errorc @ s" e Zd ZdZddd�Zdd� ZdS )� StatusMessagezXGeneral StatusMessage class. All Message classes inherit this StatusMessage class. Nc C s( || _ |pt�� | _|p t�� j| _dS )a Creates a Message. Args: message_time: Time that this message was created (since Epoch). process_id: Process ID that produced this message (overridable for testing). thread_id: Thread ID that produced this message (overridable for testing). N)�time�os�getpid� process_id� threading�current_thread�ident� thread_id)�self� message_timer r � r �#/opt/gsutil/gslib/thread_message.py�__init__& s zStatusMessage.__init__c C s d| j j| j| j| jf S )�;Returns a string with a valid constructor for this message.z#%s(%s, process_id=%s, thread_id=%s))� __class__�__name__r r r �r r r r �__str__3 s ��zStatusMessage.__str__)NN)r � __module__� __qualname__�__doc__r r r r r r r s r c s* e Zd ZdZd� fdd� Zdd� Z� ZS ) �RetryableErrorMessagez�Message class for retryable errors encountered by the JSON API. This class contains information about the retryable error encountered to report to analytics collection and to display in the UI. r Nc sp t t| �j|||d� |jj| _|jjdv r>d|jj�� | _t|t �sRt|t �rZd| _ nd| _ || _ || _ dS )a� Creates a RetryableErrorMessage. Args: exception: The retryable error that was thrown. message_time: Float representing when message was created (seconds since Epoch). num_retries: The number of retries consumed so far. total_wait_sec: The total amount of time waited so far in retrying. process_id: Process ID that produced this message (overridable for testing). thread_id: Thread ID that produced this message (overridable for testing). �r r )�socket�_socketZSocketTFN)�superr r r r � error_typer � capitalize� isinstance�apitools_service_error�six_service_error�is_service_error� num_retries�total_wait_sec)r � exceptionr r) r* r r �r r r r A s � �z RetryableErrorMessage.__init__c C s& d| j j| j| j| j| j| j| jf S )r zQ%s(%s(), num_retries=%s, total_wait_sec=%s, time=%s, process_id=%s, thread_id=%s))r r r# r) r* r r r r r r r r k s ��z RetryableErrorMessage.__str__)r r NN�r r r r r r � __classcell__r r r, r r : s �*r c s e Zd ZdZ� fdd�Z� ZS )� FinalMessagezbCreates a FinalMessage. A FinalMessage simply indicates that we have finished our operation. c s t t| ��|� dS )z�Creates a FinalMessage. Args: message_time: Float representing when message was created (seconds since Epoch). N)r" r/ r �r r r, r r r y s zFinalMessage.__init__�r r r r r r. r r r, r r/ s s r/ c s e Zd ZdZ� fdd�Z� ZS )�MetadataMessagez�Creates a MetadataMessage. A MetadataMessage simply indicates that a metadata operation on a given object has been successfully done. The only passed argument is the time when such operation has finished. c s t t| ��|� dS )z�Creates a MetadataMessage. Args: message_time: Float representing when message was created (seconds since Epoch). N)r" r2 r r0 r, r r r � s zMetadataMessage.__init__r1 r r r, r r2 � s r2 c sV e Zd ZdZdZdZdZdZdZdZ dZ d Z d Z d Z d Zd� fdd� Zdd� Z� ZS )� FileMessagez�Marks start or end of an operation for a file, cloud object or component. This class contains general information about each file/component. With that, information such as total size and estimated time remaining may be calculated beforehand. � � � � � � � � � � � NFc sD t t| �j|| | d� || _|| _|| _|| _|| _|| _|| _ dS )a� Creates a FileMessage. Args: src_url: FileUrl/CloudUrl representing the source file. dst_url: FileUrl/CloudUrl representing the destination file. message_time: Float representing when message was created (seconds since Epoch). size: Total size of this file/component, in bytes. finished: Boolean to indicate whether this is starting or finishing a file/component transfer. component_num: Component number, if dealing with a component. message_type: Type of the file/component. bytes_already_downloaded: Specific field for resuming downloads. When starting a component download, it tells how many bytes were already downloaded. process_id: Process ID that produced this message (overridable for testing). thread_id: Thread ID that produced this message (overridable for testing). r N) r" r3 r �src_url�dst_url�size� component_num�finished� message_type�bytes_already_downloaded) r r? r@ r rA rC rB rD rE r r r, r r r � s �zFileMessage.__init__c C s6 d| j j| j| j| j| j| j| j| j| j | j | j f S )r z�%s('%s', '%s', %s, size=%s, finished=%s, component_num=%s, message_type=%s, bytes_already_downloaded=%s, process_id=%s, thread_id=%s)) r r r? r@ r rA rC rB rD rE r r r r r r r � s ��zFileMessage.__str__)NFNNNNN)r r r r Z FILE_DOWNLOADZ FILE_UPLOADZFILE_CLOUD_COPYZFILE_LOCAL_COPYZFILE_DAISY_COPYZ FILE_REWRITEZ FILE_HASHZCOMPONENT_TO_UPLOADZEXISTING_COMPONENTZCOMPONENT_TO_DOWNLOADZEXISTING_OBJECT_TO_DELETEr r r. r r r, r r3 � s* �*r3 c s* e Zd ZdZd� fdd� Zdd� Z� ZS )�ProgressMessagez�Message class for a file/object/component progress. This class contains specific information about the current progress of a file, cloud object or single component. Nc sB t t| ��|� || _|| _|| _|| _|| _||k| _|| _ dS )aa Creates a ProgressMessage. Args: size: Integer for total size of this file/component, in bytes. processed_bytes: Integer for number of bytes already processed from that specific component, which means processed_bytes <= size. src_url: FileUrl/CloudUrl representing the source file. message_time: Float representing when message was created (seconds since Epoch). dst_url: FileUrl/CloudUrl representing the destination file, or None for unary operations like hashing. component_num: Indicates the component number, if any. operation_name: Name of the operation that is being made over that component. process_id: Process ID that produced this message (overridable for testing). thread_id: Thread ID that produced this message (overridable for testing). N) r" rF r rA �processed_bytesrB r? r@ rC �operation_name) r rA rG r? r r@ rB rH r r r, r r r � s zProgressMessage.__init__c C sV | j rd| j nd}| jr$d| j nd}d| jj| j| j| j| j|| j|| j | j f S )r z'%s'Nzb%s(%s, %s, '%s', %s, dst_url=%s, component_num=%s, operation_name=%s, process_id=%s, thread_id=%s)) r@ rH r r rA rG r? r rB r r )r �dst_url_stringZoperation_name_stringr r r r s ��� ��zProgressMessage.__str__)NNNNNr- r r r, r rF � s �%rF c s( e Zd ZdZ� fdd�Zdd� Z� ZS )�SeekAheadMessagea Message class for results obtained by SeekAheadThread(). It estimates the number of objects and total size in case the task_queue cannot hold all tasks at once (only used in large operations). This class contains information about all the objects yet to be processed. c s t t| ��|� || _|| _dS )aI Creates a SeekAheadMessage. Args: num_objects: Number of total objects that the SeekAheadThread estimates. size: Total size corresponding to the sum of the size of objects iterated by SeekAheadThread. message_time: Float representing when message was created (seconds since Epoch). N)r" rJ r � num_objectsrA )r rK rA r r, r r r $ s zSeekAheadMessage.__init__c C s" d| j j| j| j| j| j| jf S )r z+%s(%s, %s, %s, process_id=%s, thread_id=%s))r r rK rA r r r r r r r r 2 s ��zSeekAheadMessage.__str__r- r r r, r rJ s rJ c s* e Zd ZdZd� fdd� Zdd� Z� ZS )�ProducerThreadMessagea% Message class for results obtained by calculations made on ProducerThread. It estimates the number of objects and total size currently dealty by task_queue. If the task_queue cannot support all objects at once, the SeekAheadThread will be responsible for sending an accurate message. Fc s& t t| ��|� || _|| _|| _dS )a^ Creates a SeekAheadMessage. Args: num_objects: Number of total objects that the task_queue has. size: Total size corresponding to the sum of the size of objects iterated by the task_queue message_time: Float representing when message was created (seconds since Epoch). finished: Boolean to indicate whether this is the final message from the ProducerThread. The difference is that this message displays the correct total size and number of objects, whereas the previous ones were periodic (on the number of files) updates. N)r" rL r rK rA rC )r rK rA r rC r, r r r A s z ProducerThreadMessage.__init__c C s d| j j| j| j| j| jf S )r z%s(%s, %s, %s, finished=%s))r r rK rA r rC r r r r r T s ��z ProducerThreadMessage.__str__)Fr- r r r, r rL 9 s rL c s( e Zd ZdZ� fdd�Zdd� Z� ZS )�PerformanceSummaryMessagez�Message class to log PerformanceSummary parameters. This class acts as a relay between a multiprocess/multithread situation and the global status queue, from which the PerformanceSummary info gets consumed. c s t t| �j|ddd� || _dS )z�Creates a PerformanceSummaryMessage. Args: message_time: Float representing when message was created (seconds since Epoch). uses_slice: True if the command uses slice parallelism. Nr )r" rM r � uses_slice)r r rN r, r r r b s �z"PerformanceSummaryMessage.__init__c C s d| j j| j| jf S )r z %s(%s, %s))r r r rN r r r r r o s �z!PerformanceSummaryMessage.__str__r- r r r, r rM [ s rM )r � __future__r r r r r r �apitools.base.py.exceptionsr r& Zsix.moves.http_clientr r' �objectr r r/ r2 r3 rF rJ rL rM r r r r �<module> s" 9N9 "