/
opt
/
gsutil
/
gslib
/
utils
/
__pycache__
/
up file
home
a +(Wg� � @ s d Z ddlmZ ddlmZ ddlmZ ddlmZ ddlZddlZddlZej rVe Z g d�Z g d�Z d d � Zd Zd e Zd e Zd e Zd e Zde Ze d�Ze� \ZZdd� Zdd� Zdd� Zdd� Zdd� Z d'dd �Z d d � Z d d � Z d!d"� fd#d$�Z d%d&� Z!dS )(zCShared utility methods that calculate, convert, and simplify units.� )�absolute_import)�print_function)�division)�unicode_literalsN))r �B�bit)� ZKiBZKibit�K)� ZMiBZMibit�M)� ZGiBZGibit�G)�( ZTiBZTibit�T)�2 ZPiBZPibit�P)�< ZEiBZEibit�E))� �k)� �m)� �b)� �t)� �qc C st d} g }i }t t�D ]>\}}dd� t|�dd� D �}|D ] }|||< q:|�|� q| d�|�; } t�| �}||fS )z6Creates a suffix regex for human-readable byte counts.z'(?P<num>\d*\.\d+|\d+)\s*(?P<suffix>%s)?c S s g | ] }|� � �qS � )�lower)�.0�sr r �$/opt/gsutil/gslib/utils/unit_util.py� <listcomp>9 � z(_GenerateSuffixRegex.<locals>.<listcomp>� N�|)� enumerate� _EXP_STRINGS�list�extend�join�re�compile)Zhuman_bytes_re�suffixesZ suffix_to_si�i�siZ si_suffixes�suffixZmatcherr r r" �_GenerateSuffixRegex3 s r2 i � � r i�Q c C sX d}|d t t�k r6| dt|d d kr6|d7 }q|tt| �dt| d d�fS )Nr r% r3 � @)�lenr( �round�float)�numr/ r r r" �_RoundToNearestExponentQ s ( r: c C s |dk r d}t | �t |� S )a Calculates throughput and checks for a small total_elapsed_time. Args: total_bytes_transferred: Total bytes transferred in a period of time. total_elapsed_time: The amount of time elapsed in seconds. Returns: The throughput as a float. g{�G�z�?)r8 )Ztotal_bytes_transferredZtotal_elapsed_timer r r" �CalculateThroughputX s r; c C sF t t�D ]4\}}| d| krdt| �d| }|| S qt| �S )a� Creates a shorter string version for a given number of objects. Args: num: The number of objects to be shortened. Returns: shortened string version for this number. It takes the largest scale (thousand, million or billion) smaller than the number and divides it by that scale, indicated by a suffix with one decimal place. This will thus create a string of at most 6 characters, assuming num < 10^18. Example: 123456789 => 123.4m r z%.1lf)�reversed�_EXP_TEN_STRINGr8 �str)r9 Z divisor_expr1 �quotientr r r" � DecimalShortg s r@ c C s | | }| | dkr |d7 }|S )a Returns ceil(dividend / divisor). Takes care to avoid the pitfalls of floating point arithmetic that could otherwise yield the wrong result for large numbers. Args: dividend: Dividend for the operation. divisor: Divisor for the operation. Returns: Quotient. r r% r )ZdividendZdivisorr? r r r" � DivideAndCeilz s rA c C sp | � � } t�| �}|r`t|�d��}|�d�rPtt|�d� d }|d| 9 }tt|��}|S t d| ��dS )z�Tries to convert a human-readable string to a number of bytes. Args: human_string: A string supplied by user, e.g. '1M', '3 GiB'. Returns: An integer containing the number of bytes. Raises: ValueError: on an invalid string. r9 r1 r r5 z!Invalid byte string specified: %sN) r �MATCH_HUMAN_BYTES�matchr8 �groupr( � SUFFIX_TO_SI�intr7 � ValueError)Z human_stringr r9 Zpowerr r r" �HumanReadableToBytes� s rH r% c C s� t | ��� }tttd| t|d � ���}|dkrRd|rFdd| nd |d< n8t|�}|r�|d|| � d ||| d� }||d< d�|�S )a Creates a human readable format for bytes with fixed decimal places. Args: number: The number of bytes. decimal_places: The number of decimal places. Returns: String representing a readable format for number with decimal_places decimal places. r r �0�.� N� )�MakeHumanReadable�splitr> rF r7 r8 r6 r+ )�numberZdecimal_placesZ number_formatr9 Z num_lengthr r r" � HumanReadableWithDecimalPlaces� s" ��� �rP c C s t | �\}}d|t| d f S )z�Generates human readable string for a number of bits. Args: num: The number, in bits. Returns: A string form of the number using bit size abbreviations (kbit, Mbit, etc.) �%g %sr3 �r: r( �r9 r/ Z rounded_valr r r" �MakeBitsHumanReadable� s rT c C s t | �\}}d|t| d f S )z�Generates human readable string for a number of bytes. Args: num: The number, in bytes. Returns: A string form of the number using size abbreviations (KiB, MiB, etc.). rQ r% rR rS r r r" rM � s rM c C s | S )Nr )�xr r r" �<lambda>� r$ rV c C s| | sdS t | �d | }t�|�}t�|�}||krD|| t|� �S || t|� �|| }|| t|� �|| }|| S )a� Find the percentile of a list of values. Taken from: http://code.activestate.com/recipes/511478/ Args: values: a list of numeric values. Note that the values MUST BE already sorted. percent: a float value from 0.0 to 1.0. key: optional key function to compute value from each element of the list of values. Returns: The percentile of the values. Nr% )r6 �math�floor�ceilrF )�values�percent�keyr �f�cZd0Zd1r r r" � Percentile� s r_ c C sz t t| ��} | d }|dkr*dt|d� S | d| 8 } | d }| d| 8 } | }td| �d td| � d td| � S )aD Creates a standard version for a given remaining time in seconds. Created over using strftime because strftime seems to be more suitable for a datetime object, rather than just a number of seconds remaining. Args: remaining_time: The number of seconds remaining as a float, or a string/None value indicating time was not correctly calculated. Returns: if remaining_time is a valid float, %H:%M:%D time remaining format with the nearest integer from remaining_time (%H might be higher than 23). Else, it returns the same message it received. i �d z%d+ hrsi� r z%02d�:)rF r7 �minr> )Zremaining_time�hours�minutes�secondsr r r" � PrettyTime� s �rf )r% )"�__doc__� __future__r r r r rW r, �six�PY3rF Zlongr( r= r2 �ONE_KIB�ONE_MIB�ONE_GIBZTWO_MIBZ EIGHT_MIBZTEN_MIBZSECONDS_PER_DAYrE rB r: r; r@ rA rH rP rT rM r_ rf r r r r" �<module> s<