/
opt
/
gsutil
/
gslib
/
utils
/
__pycache__
/
up file
home
a +(Wg�J � @ s� d Z ddlmZ ddlmZ ddlmZ ddlmZ ddlmZ ddlZddl Z ddl Z ddl Z ddl Z ddl Z ddlmZ dd lmZ dd lmZ dd lmZ dd lmZ dd lmZ es�ddlZddlZe jr�e Z dZ dZ dZ dZ!dZ"dZ#dZ$dZ%e �&d�Z'dZ(dZ)dZ*dZ+dZ,dZ-dZ.d Z/d Z0da1e2� a3G d d � d e4�Z5G d d!� d!e4�Z6d"d#� Z7d$d%� Z8dAd'd(�Z9d)d*� Z:d+d,� Z;d-d.� Z<e$e$e%fd/d0�Z=dBd1d2�Z>d3d4� Z?d5d6� Z@d7d8� ZAd9d:� ZBd;d<� ZCd=d>� ZDd?d@� ZEdS )Cz Helper file for POSIX methods.� )�absolute_import)�print_function)�division)�unicode_literals)�timegmN)�CommandException)�UTC)�CreateCustomMetadata)� GetValueFromObjectCustomMetadata)� IS_WINDOWS)�SECONDS_PER_DAYzgoog-reserved-file-atimezgoog-reserved-posix-gidzgoog-reserved-posix-modezgoog-reserved-file-mtimezgoog-reserved-posix-uid���z ^[0-7]{3}$� � �@ � � � � � � c @ s$ e Zd ZdZeeeedfdd�ZdS )�POSIXAttributesz1Class to hold POSIX attributes for a file/object.Nc C s. || _ || _|| _|| _t|r"|nt�| _dS )a6 Constructor for POSIXAttributes class which holds relevant data. Args: atime: The access time of the file/object. mtime: The modification time of the file/object. uid: The user ID that owns the file. gid: The group ID that the user is in. mode: An instance of POSIXMode. N)�atime�mtime�uid�gid� POSIXMode�NA_MODE�mode)�selfr r r r r � r �%/opt/gsutil/gslib/utils/posix_util.py�__init__V s zPOSIXAttributes.__init__)�__name__� __module__� __qualname__�__doc__�NA_TIME�NA_IDr" r r r r! r S s �r c @ s e Zd Zdd� ZdS )r c C s || _ d S )N)� permissions)r r) r r r! r" n s zPOSIXMode.__init__N)r# r$ r% r" r r r r! r l s r c C s t t| �dd� �S )z7Converts a base-10 mode integer from os.stat to base-8.���N)�int�oct�r r r r! �ConvertModeToBase8r s r. c C s� t � }t| tt�\}}zJt|�}|r<|tkr<td|� t}n"|tt�� �t kr^td|� t}W n t y� t d|� t}Y n0 ||_ t | t ||� t | t||� t| tt�\}}|r�t�|�r�ztt|��|_W n t y� t d|� Y n0 |S )aA Parses the POSIX attributes from the supplied metadata. Args: obj_metadata: The metadata for an object. url_str: File/object path that provides context if a warning is thrown. Returns: A POSIXAttribute object with the retrieved values or a default value for any attribute that could not be found. r r )r r � ATIME_ATTRr' �long�WarnNegativeAttribute�timer �WarnFutureTimestamp� ValueError�WarnInvalidValuer �DeserializeIDAttribute�GID_ATTR�UID_ATTR� MODE_ATTRr � MODE_REGEX�matchr r+ r )� obj_metadata�url_str� posix_attrs�foundr r r r r! �+DeserializeFileAttributesFromObjectMetadatay s6 � � r@ Fc C s� | j tkr tt| j i|d� |r�| jtkr<tt| ji|d� | jtkrXtt| ji|d� | j tkrttt | j i|d� | j j t kr�tt| j j i|d� dS )a+ Takes a POSIXAttributes object and serializes it into custom metadata. Args: posix_attrs: A POSIXAttributes object. custom_metadata: A custom metadata object to serialize values into. preserve_posix: Whether or not to preserve POSIX attributes other than mtime. )�entries�custom_metadataN)r r' r � MTIME_ATTRr r/ r r( r8 r r7 r r) r r9 )r> rB �preserve_posixr r r! �'SerializeFileAttributesToObjectMetadata� s* � � � � �rE c C sv |� d�d }t| |t�\}}z&t|�}|rB|tkrBt||� t}W n tyd t||� t}Y n0 t|||� dS )a Parses the POSIX attributes from the supplied metadata into posix_attrs. Args: obj_metadata: The metadata for an object. attr: Either GID_ATTR or UID_ATTR. url_str: File/object path that provides context if a warning is thrown. posix_attrs: POSIXAttribute object. �-r N)�splitr r( r+ r1 r4 r5 �setattr)r< �attrr= r> � attr_namer? �valr r r! r6 � s r6 c C s� t � } | tk} |tk} |tk} |tk}|tk}|tk}|tk}|tk}|tk}| tk}| rd| sd| | _| rr|sr|| _|r�|s�|| _|r�|s�|| _|r�|s�|| j_ | | r�| p�| r�| p�|r�| p�|r�| p�|o�| fS )a; Checks whether an update for any POSIX attribute is needed. Args: src_atime: The source access time. dst_atime: The destination access time. src_mtime: The source modification time. dst_mtime: The destination modification time. src_uid: The source user ID. dst_uid: The destination user ID. src_gid: The source group ID. dst_gid: The destination group ID. src_mode: The source mode. dst_mode: The destination mode. Returns: A tuple containing a POSIXAttribute object and a boolean for whether an update was needed. ) r r' r( r r r r r r r) )Z src_atimeZ dst_atimeZ src_mtimeZ dst_mtimeZsrc_uidZdst_uidZsrc_gidZdst_gidZsrc_modeZdst_moder> Z has_src_atimeZ has_dst_atimeZ has_src_mtimeZ has_dst_mtimeZ has_src_uidZ has_dst_uidZ has_src_gidZ has_dst_gidZ has_src_modeZ has_dst_moder r r! �NeedsPOSIXAttributeUpdate� s< ����rL c C s. t �t| �dd� �o,| t@ p,| t@ p,| t@ S )z�Validates whether the mode is valid. In order for the mode to be valid either the user, group, or other byte must be >= 4. Args: mode: The mode as a 3-digit, base-8 integer. Returns: True/False r* N)r: r; r, �U_R�G_R�O_Rr- r r r! �ValidatePOSIXMode s &�rP c C s� t rdS |tk}t|�tk}|tk}|s4|s4|s4dS t�� dkrDdS ttt|�d��}|rl|stdd| fS ntt�}|r�zt � |� W n& t t fy� dd| |f f Y S 0 |r�zt �|� W n& t t fy� dd| |f f Y S 0 |t�� k}tt|�d�}|�s@|�r@|�r@t|t@ ��s@dd| |t|�d d � f fS |�rzt|t@ �} | | �r^d nd | |t|�d d � f fS t|�tv �r�t|t@ �} | | �r�d nd| |t|�d d � f fS |t@ �r�dS |�s�|�s�|�r�dS dd | fS )a� Validates that the user has file access if uid, gid, and mode are applied. Args: url_str: The path to the object for which this is validating. uid: A POSIX user ID. gid: A POSIX group ID. mode: A 3-digit, number representing POSIX permissions, must be in base-8. Returns: A (bool, str) tuple, True if and only if it's safe to copy the file, and a string containing details for the error. )T� r r Fz$Mode for %s won't allow read access.z3UID for %s doesn't exist on current system. uid: %dz3GID for %s doesn't exist on current system. gid: %dz?Insufficient access with uid/gid/mode for %s, gid: %d, mode: %sr* NrQ z?Insufficient access with uid/gid/mode for %s, uid: %d, mode: %sz"There was a problem validating %s.)r r( r+ r �os�geteuidrP �str�SYSTEM_POSIX_MODE�pwd�getpwuid�KeyError� OverflowError�grp�getgrgid�getuid�boolrM r, � USER_GROUPSrN rO ) r= r r r Z uid_presentZ gid_presentZ mode_presentZ mode_validZuid_is_current_user�validr r r! � ValidateFilePermissionAccess sj � � � �� �� r` c C sN |du rt d| ���zt|ttd�\}}t|ttd�\}}t|ttd�\}} t|ttd�\} } t|tt d�\} } |r�t |�}|s�t � | �j }t �| ||f� W dS n |r�t|j�}t �| ||f� |s�W dS |r�t |�}|tkr�|tkr�t �| ||f� nb|tk�r.|tk�r.t � | �j}t �| ||f� n0|tk�r^|tk�r^t � | �j }t �| ||f� t�rjW dS |�r�t �� dk�r�t| �} nt} | �r�t| �} | tk�r�| tk�r�t �| | | � nF| tk�r�| tk�r�t �| | d� n"| tk�r| tk�rt �| d| � | �r$tt| �d�} t �| | � W n" t�yH t d|j ��Y n0 dS )a� Parses POSIX attributes from obj_metadata and sets them. Attributes will only be set if they exist in custom metadata. This function should only be called after ValidateFilePermissionAccess has been called for the specific file/object so as not to orphan files. Args: path: The local filesystem path for the file. Valid metadata attributes will be set for the file located at path, some attributes will only be set if preserve_posix is set to True. obj_metadata: The metadata for the file/object. is_rsync: Whether or not the caller is the rsync command. Used to determine if timeCreated should be used. preserve_posix: Whether or not all POSIX attributes should be set. Nz"obj_metadata cannot be None for %s)� default_valuer r r z#Check POSIX attribute values for %s)r r r/ r' rC r8 r( r7 r9 r r0 rR �stat�st_atime�utime�ConvertDatetimeToPOSIX� timeCreated�st_mtimer rS r+ �chownrT �chmodr4 �name)�pathr< �is_rsyncrD Zfound_atr Zfound_mtr Z found_uidr Z found_gidr Z found_moder Z atime_tmpZ mtime_tmpr r r! �ParseAndSetPOSIXAttributeso s� � � � � � �rm c C s t �� �d|| � dS )z�Logs if an attribute has a negative value. Args: attr_name: The name of the attribute to log. url_str: The path of the file for context. z$%s has a negative %s in its metadataN��logging� getLogger�warn�rJ r= r r r! r1 � s �r1 c C s t �� �d|| � dS )��Logs if an attribute has an invalid value. Args: attr_name: The name of the attribute to log. url_str: The path of the file for context. z$%s has an invalid %s in its metadataNrn rr r r r! r5 � s �r5 c C s t �� �d|| � dS )rs z5%s has an %s more than 1 day from current system timeNrn rr r r r! r3 � s �r3 c C s t t| jt� d��� ��S )a Converts a datetime object to UTC and formats as POSIX. Sanitize the timestamp returned in dt, and put it in UTC format. For more information see the UTC class. Args: dt: A Python datetime object. Returns: A POSIX timestamp according to UTC. )�tzinfo)r0 r �replacer � timetuple)�dtr r r! re � s re c C sD t r dadS d} t�d�}t�|� | | }t|d@ �dd� adS )z.Records the default POSIX mode using os.umask.Z666Ni� � i� r* )r rU rR �umaskr, )Zmax_permissionsZ current_umaskr r r r! �InitializeDefaultMode� s rz c sH t rdS t�� } t�| �j� tt�| �jg� fdd�t� � D � �a dS )z�Initializes the set of groups that the user is in. Should only be called if the flag for preserving POSIX attributes is set. Nc s g | ]}� |j v r|j�qS r )�gr_mem�gr_gid)�.0�g�Z user_namer r! � <listcomp> � z(InitializeUserGroups.<locals>.<listcomp>) r rR r\ rV rW �pw_name�set�pw_gidrZ �getgrallr^ )�user_idr r r! �InitializeUserGroups s ��r� c C s t � t� dS )z<Initializes POSIX data. Run once at the beginning of a copy.N)rz r� r r r r! �InitializePreservePosixData" s r� )F)FF)Fr& � __future__r r r r �calendarr �getpassro rR �rer2 �six�gslib.exceptionr Z gslib.tz_utcr �gslib.utils.metadata_utilr r �gslib.utils.system_utilr �gslib.utils.unit_utilr rZ rV �PY3r+ r0 r/ r7 r9 rC r8 r' r( r �compiler: rM �U_WZU_XrN ZG_WZG_XrO ZO_WZO_XrU r� r^ �objectr r r. r@ rE r6 rL rP r` rm r1 r5 r3 re rz r� r� r r r r! �<module> s| , � 1X � _