golden hour
/opt/saltstack/salt/lib/python3.10/site-packages/salt/cloud/__pycache__
⬆️ Go Up
Upload
File/Folder
Size
Actions
__init__.cpython-310.pyc
47.6 KB
Del
OK
cli.cpython-310.pyc
8.63 KB
Del
OK
exceptions.cpython-310.pyc
2.18 KB
Del
OK
libcloudfuncs.cpython-310.pyc
9.46 KB
Del
OK
Edit: __init__.cpython-310.pyc
o �xeb � @ s� d Z ddlZddlZddlZddlZddlZddlZddlZddlZddl m Z ddl Z ddl Z ddlZ ddlZ ddlZ ddlZ ddlZ ddlZ ddlZ ddlZ ddlZ ddlZ ddlZ ddlZ ddlmZm Z m Z m Z ddl m Z zddl!Z"W n e#y� zddl$Z%W n e#y� Y nw Y nw e�&e'�Z(dd� Z) d dd �Z*G d d � d �Z+G d d � d �Z,G dd� de,�Z-dd� Z.d dd�Z/d dd�Z0d dd�Z1dd� Z2dd� Z3d d � Z4dS ) z` The top level interface used to translate configuration data back to the correct cloud modules � N)�groupby)�SaltCloudConfigError�SaltCloudException�SaltCloudNotFound�SaltCloudSystemExit)�compile_templatec s � fdd�}|S )z(Warning, this is a picklable decorator !c s$ | |d< d}z� |i |��}| � d� W |S ty? } z t�� }| � d� | � d� | � d�||�� W Y d}~|S d}~w tyh } z t�� }| � d� | � d� | � d�||�� W Y d}~|S d}~w ty� } z t�� }| � d� | � d � | � d�||�� W Y d}~|S d}~ww ) z7called with [queue, args, kwargs] as first optional arg�queueN�END� KEYBOARDINTzKeyboard interruptz{} {} �ERROR� Exceptionz System exit)Zput�KeyboardInterrupt� traceback� format_exc�formatr � SystemExit)r �args�kwargs�ret�ex�trace��func� �G/opt/saltstack/salt/lib/python3.10/site-packages/salt/cloud/__init__.py�_call3 s8 � �� �� ��zcommunicator.<locals>._callr )r r r r r � communicator0 s r c s6 � si � |sd}|st �|�}�st �� }|�� �|dur0|s0d�| �} t�| � td�| ���|durF� �fdd�|D �} |j| | d�} n |� | �|� g�} �� � } | d v r��� � } �� � }d �| �} | d �|�7 } t�| � |� � |� � td�| ���| d v s�|r�|| �r�|� � |� � | S t�d � qP)a' Manage a multiprocessing pool - If the queue does not output anything, the pool runs indefinitely - If the queue returns KEYBOARDINT or ERROR, this will kill the pool totally calling terminate & join and ands with a SaltCloudSystemExit exception notifying callers from the abnormal termination - If the queue returns END or callback is defined and returns True, it just join the process and return the data. target the function you want to execute in multiprocessing pool pool object can be None if you want a default pool, but you ll have then to define pool_size instead pool_size pool size if you did not provide yourself a pool callback a boolean taking a string in argument which returns True to signal that 'target' is finished and we need to join the pool queue A custom multiprocessing queue in case you want to do extra stuff and need it later in your program args positional arguments to call the function with if you don't want to use pool.map mapped_args a list of one or more arguments combinations to call the function with e.g. (foo, [[1], [2]]) will call:: foo([1]) foo([2]) kwargs kwargs to give to the function in case of process Attention, the function must have the following signature: target(queue, *args, **kw) You may use the 'communicator' decorator to generate such a function (see end of this file) � NzdWe are called to asynchronously execute {} but we do no have anything to execute, weird, we bail outzException caught {}c s g | ]}�|g� g�qS r r )�.0�arg�r r r r � <listcomp>� s z"enter_mainloop.<locals>.<listcomp>)r �iterableT)r r z Caught {}, terminating workers z TRACE: {} )r g �?)�multiprocessing�PoolZManagerZQueuer �log�errorr �mapZapply�get� terminate�join�close�time�sleep)�targetZ mapped_argsr r �pool� pool_size�callbackr Zmanager�msgr" r �testZtype_r r r r �enter_mainloopN sJ 9 � ��r4 c @ s� e Zd ZdZd'dd�Zdd� Zdd� Zd(d d �Zd(d d �Zd(d d�Z d)dd�Z d*dd�Z d+dd�Z d,dd�Z d(dd �Zd(d d �Zd d � Zd!d"� Zd#d$� Z d-d%d&�ZdS ).� CloudClientz5 The client class to wrap cloud interactions Nc C sf |r|| _ ntj�|�| _ | j d g}tjj�|tjj�� � |r�|� di �� � D ]\}}|d }i |d< | j d � |||ii� q)|� di �� � D ]@\}} | d � d�d }t t| j d | �� ��}d�||�| d< || d < | j d � || i� | j d | | d � || i� qJ|� d i �� � D ]\}} d | j vr�i | j d < | | j d |< q�| j � |� d S d S ) NZcachedir� providers�driver�profiles�provider�:r �{}:{}�profile�maps)�opts�salt�configZ cloud_config�utilsZverifyZ verify_env�userZget_user�pop�items�update�split�next�iter�keysr ) �self�pathr> Z config_dirZpillarsZv_dirs�namer9 r7 r< Zmap_dctr r r �__init__� s2 � �zCloudClient.__init__c K s� t jj�� }|�| j�� � d|d< d|d< d|d< d|d< d|d< d|d < d |v r1|�|d � |�|� |�d d �}|rz|�d i ��� }dd� |D �D ] }||ksX|�|� qMdd� |�� D �}|rz|�di �}|�� D ] }||vry|�|� qn|S )zi Set the opts dict to defaults and allow for opts to be overridden in the kwargs F�parallelZkeep_tmpT�deployZupdate_bootstrap�show_deploy_args� Z script_argsr r< Nr8 c S s g | ]}|�qS r r �r �ar r r r! � s z.CloudClient._opts_defaults.<locals>.<listcomp>c S s. g | ]}|� d d�r|� d d��d�d �qS )r9 rQ r: r )r( rF rR r r r r! � s ��r6 ) r? r@ ZDEFAULT_CLOUD_OPTS�copyrE r> r( rC �values) rJ r r> r< Z tmp_profilesZ_profiler6 Z _providersr9 r r r �_opts_defaults� s8 �� �zCloudClient._opts_defaultsc C s8 t | |�}tjj�||�}||�dd�i |�di ���S )zG Pass the cloud function and low data structure to run r r r )�getattrr? rA r Z format_callr( )rJ �fun�lowZl_funZf_callr r r rY � s zCloudClient.lowc C �$ t j�| �� �}t jj�|�|��S )zF List all available sizes in configured cloud systems )r? �cloud�MaprV rA �data�simple_types_filter� size_list�rJ r9 �mapperr r r � list_sizes � zCloudClient.list_sizesc C rZ )zG List all available images in configured cloud systems )r? r[ r\ rV rA r] r^ � image_listr` r r r � list_images rc zCloudClient.list_imagesc C rZ )zJ List all available locations in configured cloud systems )r? r[ r\ rV rA r] r^ � location_listr` r r r �list_locations rc zCloudClient.list_locations� list_nodesc C �$ t j�| �� �}d|jd< |�|�S )z2 Query basic instance information rh �selected_query_option�r? r[ r\ rV r> �map_providers_parallel�rJ Z query_typera r r r �query � zCloudClient.query�list_nodes_fullc C ri )z0 Query all instance information rp rj rk rm r r r � full_query$ ro zCloudClient.full_query�list_nodes_selectc C ri )�3 Query select instance information rr rj rk rm r r r � select_query, ro zCloudClient.select_query�list_nodes_minc C ri )rs ru rj rk rm r r r � min_query4 ro zCloudClient.min_queryc K sV |si }||d< t j�| jdi |���}t|t�r |�d�}t jj� |j |||d��S )aY Pass in a profile to create, names is a list of vm names to allocate vm_overrides is a special dict that will be per node options overrides Example: .. code-block:: python >>> client= salt.cloud.CloudClient(path='/etc/salt/cloud') >>> client.profile('do_512_git', names=['minion01',]) {'minion01': {'backups_active': 'False', 'created_at': '2014-09-04T18:10:15Z', 'droplet': {'event_id': 31000502, 'id': 2530006, 'image_id': 5140006, 'name': 'minion01', 'size_id': 66}, 'id': '2530006', 'image_id': '5140006', 'ip_address': '107.XXX.XXX.XXX', 'locked': 'True', 'name': 'minion01', 'private_ip_address': None, 'region_id': '4', 'size_id': '66', 'status': 'new'}} r< �,)� vm_overridesNr ) r? r[ r\ rV � isinstance�strrF rA r] r^ � run_profile)rJ r< �namesrx r ra r r r r< < s �zCloudClient.profilec K sN i }|r||d< |� |� tj�| jdi |���}|�� }tjj�|� |��S )z" To execute a map r' Nr ) rE r? r[ r\ rV �map_datarA r] r^ �run_map)rJ rK r �kwargra �dmapr r r �map_runf s zCloudClient.map_runc C s< t j�| jdd��}t|t�r|�d�}t jj� |� |��S )�' Destroy the named VMs T)�destroyrw ) r? r[ r\ rV ry rz rF rA r] r^ r� )rJ r| ra r r r r� r s zCloudClient.destroyc K s� t j�| �� �}| jd }||v r!|d�tt|| �� ���7 }ndS t |t �r-|� d�}i }|D ]"}|� � }||d< ||d< d|d< ||d < t j j�|�|��||< q1|S ) aM Create the named VMs, without using a profile Example: .. code-block:: python client.create(provider='my-ec2-config', names=['myinstance'], image='ami-1624987f', size='t1.micro', ssh_username='ec2-user', securitygroup='default', delvol_on_destroy=True) r6 �:{}Frw rL r7 Nr< r9 )r? r[ r\ rV r> r rG rH rI ry rz rF rT rA r] r^ �create) rJ r9 r| r ra r6 r rL �vm_r r r r� { s zCloudClient.createc K s� t j�| �� �}|�� }||v r |d�tt|| �� ���7 }ndS t |t �r,|� d�}i }|D ]"}|� � } || d< || d< d| d< || d< t j j�|�| ��||< q0|S ) a� Perform actions with block storage devices Example: .. code-block:: python client.extra_action(names=['myblock'], action='volume_create', provider='my-nova', kwargs={'voltype': 'SSD', 'size': 1000} ) client.extra_action(names=['salt-net'], action='network_create', provider='my-nova', kwargs={'cidr': '192.168.100.0/24'} ) r� Frw rL r9 Nr< �action)r? r[ r\ rV rl r rG rH rI ry rz rF rT rA r] r^ �extras) rJ r| r9 r� r ra r6 r rL �extra_r r r � extra_action� s zCloudClient.extra_actionc C s| |du ri }t j�| jd||d�|���}|r |r td��|g}|r/|s/|| jd< |�||�S |r:|s:|�|||�S td��)aD Execute a single action via the cloud plugin backend Examples: .. code-block:: python client.action(fun='show_instance', names=['myinstance']) client.action(fun='show_image', provider='my-ec2-config', kwargs={'image': 'ami-10314d79'} ) N)r� r| zMPlease specify either a list of 'names' or a single 'instance', but not both.r� zTEither an instance (or list of names) or a provider must be specified, but not both.r )r? r[ r\ rV r r> � do_action� do_function)rJ rX Zcloudmapr| r9 �instancer ra r r r r� � s" � �zCloudClient.action)NNNN�N)rh )rp )rr )ru )NNNNNN)�__name__� __module__� __qualname__�__doc__rM rV rY rb re rg rn rq rt rv r< r� r� r� r� r� r r r r r5 � s0 * * &$�r5 c @ s� e Zd ZdZdd� Zdd� Zdd� Zdd � Zd6d d �Zd6dd�Z d7dd�Z dd� Z d8dd�Z d8dd�Z d8dd�Zd8d d �Zd8d d �Zd d!� Zd9d"d#�Zd$d%� Zd:d(d)�Zed*d+� �Zd,d-� Zd;d.d/�Zd0d1� Zd2d3� Zd4d5� ZdS )<�Cloudz/ An object for the creation of new VMs c C s( || _ tj�| j �| _| �� i | _d S r� )r> r? �loader�clouds�$_Cloud__filter_non_working_providers� _Cloud__cached_provider_queries�rJ r> r r r rM � s zCloud.__init__c C sV t � }| jd �� D ] \}}t|�dkr#|D ] }|�d�||�� qq |�|� q |S )z1 Return the configured providers r6 r r; )�setr> rD �len�addr )rJ r6 �alias�driversr7 r r r �get_configured_providers s z Cloud.get_configured_providersc C s |du rd}|dkr-t � }| jd �� D ]\}}|D ] }|�||f� qq|s+td��|S d|v rU|�d�\}}|| jd vsH|| jd | vrUtd�|d�| �� ����t � }| jd �� D ]\}}|D ]}|||fv rt|�||f� qeq_|s�td�|d�| �� ����|S ) z@ Get a dict describing the configured providers N�allr6 z(There are no cloud providers configured.r: z.No cloud providers matched '{}'. Available: {}�, z9No cloud providers matched '{}'. Available selections: {}) r� r> rD r� r rF r r* r� )rJ �lookupr6 r� r� r7 r r r �lookup_providers sD ��� ����zCloud.lookup_providersc C s� |du rd}|du r d}|dkrct � }t � }| jd �� D ]7\}}|�d�}|rT|�d�d }|�d�d } ||krB|�||f� n | |krM|�|| f� |�||f� q |s[td��|dkra|S |S dS ) zH Return a dictionary describing the configured profiles Nr� r8 r9 r: r r z'There are no cloud profiles configured.)r� r> rD r( rF r� r ) rJ r9 r� r8 Zprovider_profilesr� �infor6 Zgiven_prov_nameZsalt_prov_namer r r �lookup_profiles8 s0 ��zCloud.lookup_profilesrh Fc C s* |du r|| j v r| j | S i }| jd �� D ]v\}}|�� D ]m\}}d�||�}|| jvr5t�d|� q ||vr=i ||< z-tjj j | j| d� ||g�d�� | j| � || |< W d � n1 sdw Y W q t y� } ztj d|| tjd � g || |< W Y d} ~ q d} ~ ww q|| j |< |S ) z� Return a mapping of what named VMs are running on what VM providers based on what providers are defined in the configuration and VMs Tr6 �{}.{}�)Public cloud provider %s is not availabler: �Z__active_provider_name__N�=Failed to execute '%s()' while querying for running nodes: %s�Zexc_info_on_loglevel)r� r> rD r r� r% r& r? rA �context�func_globals_injectr* r �debug�logging�DEBUG) rJ rn �cached�pmapr� r� r7 �detailsrX �errr r r � map_providersX s@ � ������ zCloud.map_providersc C s> |du r|| j v r| j | S | j�� }g }| �|d �|d< |d �� D ]=\}}|}|�� D ]2\}} |�d�du rCd�|�| jv rCd}d�||�} | | jvrUt� d|� q.|� | ||||d �� q.q$i } |sh| S t |�} t � | d k rt| pud t�} tt|| d �}|D ]\}}} | s�q�|| vr�i | |< | | | |< q�| | j |< | S ) z� Return a mapping of what named VMs are running on what VM providers based on what providers are defined in the configuration and VMs Same as map_providers but query in parallel. Tr6 rj Nz{}.list_nodes_minru r� r� )rX r> rn r� r7 � )r/ )r� r> rT �_optimize_providersrD r( r r� r% r& �appendr� r# r$ �init_pool_workerr4 �!_run_parallel_map_providers_query)rJ rn r� r> Zmultiprocessing_datar� r� Z this_queryr7 r� rX �outputZ data_countr/ Z parallel_pmapr r r rl } sT ����� z Cloud.map_providers_parallelNc C sD t |t�r|g}i }i }| j||d�}|�� D ]�\}} | �� D ]\} } | |vr+||| < |r>|| jd | d �d�d vr>q | �� D ][\} } | |vrKqB| dkrfd|v rfd||d v rf| ||d d v rfqB| dkr�d|v r�d||d v r�| ||d d v r�qB||vr�i ||< | || vr�i || | < | || | | < qBq q|S )N�r� r8 r9 r: r Zec2Zaws)ry rz rl rD r> rF )rJ r| rn r� r< �matchesZhandled_drivers�mapped_providersr� r� r7 �vms�vm_namer� r r r �get_running_by_names� sH �� ��-zCloud.get_running_by_namesc C s� i }i }|� � D ]\}}|� � D ]\}}||vr i ||< ||| |< qq|� � D ]L\}}d�|�} | | jvrTt�d|� |� � D ]\}} ||vrLi ||< | || |< q@q(| j| |�} | rt| � � D ]\}} ||vrmi ||< | || |< qaq(|S )zD Return an optimized mapping of available providers z{}.optimize_providersz0The '%s' cloud driver is unable to be optimized.)rD r r� r% r� ) rJ r6 Z new_providersZprovider_by_driverr� r7 rL r] Zproviders_datarX Z prov_dataZnew_datar r r r� � s2 � �zCloud._optimize_providersr� c C �� i }| � |�}|s |S |D ]g\}}d�|�}|| jvr#t�d||� q ||vr+i ||< z-tjjj| j| d� ||g�d�� | j| � || |< W d � n1 sRw Y W q t yt } ztj d||t j d� W Y d}~q d}~ww |S )zO Return a mapping of all location data for available providers z{}.avail_locationszbThe '%s' cloud driver defined under '%s' provider alias is unable to get the locations informationr: r� N�&Failed to get the output of '%s()': %sr� �r� r r� r% r� r? rA r� r� r* r r& r� r� �rJ r� r] �lookupsr� r7 rX r� r r r rf s@ �� �����zCloud.location_listc C r� )�L Return a mapping of all image data for available providers z{}.avail_imagesz_The '%s' cloud driver defined under '%s' provider alias is unable to get the images informationr: r� Nr� r� r� r� r r r rd A �@ �� �����zCloud.image_listc C r� )r� z{}.avail_sizesz^The '%s' cloud driver defined under '%s' provider alias is unable to get the sizes informationr: r� Nr� r� r� r� r r r r_ i r� zCloud.size_listc C sP i }| � |�}|s |S |D ]\}}||vri ||< ||| vr%i || |< q |S )r� )r� )rJ r� r] r� r� r7 r r r � provider_list� s �zCloud.provider_listc C sR i }| � ||�}|s |S |D ]\}}||vri ||< ||| vr&i || |< q|S )z= Return a mapping of all configured profiles )r� )rJ r9 r� r] r� r� r7 r r r � profile_list� s �zCloud.profile_listc C s4 g }| j d �� D ]\}}|�|| �|�i� q |S )z6 Create/Verify the VMs in the VM data r8 )r> rD r� r� )rJ r r� � vm_detailsr r r � create_all� s zCloud.create_allc C s� i }t |�}| j||d�}t � }g }|�� D ].\}}|�� D ]%\} } | D ] } | |v rB|�|| | f� | jd rB|�| j| || d�� q$q q| jd r�|r�d| jv rW| jd } nt|�} t�d| � t t || d�} i }| D ]}|� |� ql|D ]5}|d }|d } |d } ||vr�i ||< | || vr�i || | < || || | | < | |v r�|� | � qvn`t�d � |D ]X\}} } d � | �}tjjj| j| d �|| g�d �� | j| | �}W d � n1 s�w Y ||vr�i ||< | || vr�i || | < ||| | | < | |v �r |� | � q�|D �]$\}} } || | | }|�s"�q| dd �|| g�| d�}tjjd|| ji d�}tj�| jd d|�d| ��}t�d� |��}tj�|��st|�stt|t ��rrd|v �rrtjj � | jd |d � �qtj�|��r�|�s�tjj � | jd tj� |�� �qtj�|��s�|�r�t|�dk�r�tjj � | jd tj� |d �� �qt d� | �� t!|�D ]\}}t d� |tj� |��� �q�t"d �}zt#|�}W n t$�y� t d � |�� Y nw z tj� |�%|��}W n t&�y Y �q�w t"d � |��}|d k�s|�'� �(d ��r+tjj � | jd |� t d!� |�� nt d"� |�� �q|�rE|�sEt)d#� d$�|����|�rQ|�rQ||d%< |S |�sXt)d&��|S )'r� r� rN )r> rL r� r7 r0 z0Destroying in parallel mode; Cloud pool size: %s�r0 r� r7 rL z Destroying in non-parallel mode.� {}.destroyr: r� N)rL r< r9 r7 �minion��default�pki_dirZminions�idz{}.*Znewnamer r zjThere are several minion keys who's name starts with '{}'. We need to ask you which one should be deleted:Tz {}: {}z,Which minion key should be deleted(number)? z '{}' is not a valid selection.zDelete '{}'? [Y/n]? rQ �yz Deleted '{}'zDid not delete '{}'z%The following VM's were not found: {}r� � Not FoundzNo machines were destroyed!)*r� r� rD r� r> r� r� r% r� r4 �_destroy_multiprocessingrE �remover r? rA r� r� r� r* r@ �get_cloud_config_value�osrK r( �glob�isfilery �dictr[ Z remove_key�basename�print� enumerate�input�int� ValueErrorrC r �lower� startswithr )rJ r| r� Z processed�matchingZvms_to_destroy� parallel_datar� r� r7 r� rL r0 � output_multipZ ret_multip�objrX r r� � minion_dict�key_fileZglobbed_key_file�idx�filenameZ selection�deleter r r r� � s ����� � �� � � � � ��� ���� ��� � �z Cloud.destroyc C s� g }| � � }i }|�� D ]\}}g ||< |D ] }||v r#|| �|� qq |�� D ]\}}d�|�} |D ]} |�| | j| | �i� q4q)|S )z& Reboot the named VMs z {}.reboot)rl rD r� r r� ) rJ r| r r� Zacts�prov�nodes�nodeZnames_rX rL r r r �rebootg s �� �z Cloud.rebootT� c C s� i }t jjd|| ji d�}|d �d�\}}d�|�}|| jvr.t�d|d |d |� dS t j�d || j�} t j�d || j�} | ry| sPd |vrPt� d |d � d |vrxd|vrxt� d|d � t j j � t j�d|| j��\} } | |d < | |d< nd|d < d|d< |�d|d �} |�d�}|�d�r�|r�||d< d|v r�d�| |d g�} | du r�d|vr�d|vr�t� d|d � t j j � t j�d|| j��\}}||d< ||d< |du r�| du r�t j j �| jd |d | � t j�d|| j�|d < z t j�d || j�|d < W n t�y Y nw z�|d �d�\}}d�|�}t j jj| j| d�||g�d �� | j| |�}W d � n 1 �s>w Y |d u�r�d | jv �r�| jd �rf| jd d!v�rft� d"� d#| jd < t�|� t�� d$ }t�� |k �r�t�t jj�}d%�| jd& �d%�dd'� �}|�t j�tj �|d ��� t j j |d(��,}|j |d d)�| jd �| jd* d+�}|�r�t� d,|� W d � nW d � n 1 �s�w Y t�� |k �sxW n t�y� } zt�!d-|d |� W Y d}~nd}~ww z| jd. }W n t�y d }Y nw | jd/ �r[| jd0 �r[|�s[t� d1| jd0 |d � t j j | jd(��}|j |d | jd0 | jd* d$ d+�}W d � n 1 �sRw Y ||d2< |S )3z$ Create a single VM r� r� r9 r: z {}.create�TCreating '%s' using '%s' as the provider cannot complete since '%s' is not availablerL NrO � make_master�masterz2There's no master defined on the '%s' VM settings.�pub_key�priv_key� Generating minion keys for '%s'�keysizer� �domainZuse_fqdnZ append_domain�.T� master_pub� master_pemz#Generating the master keys for '%s'r� �scriptr� Z inline_scriptr� FZsync_after_install)r� Zbeaconsr� ZenginesZ executors�grainsr% Zmatchers�modulesr� ZpillarZ proxymodulesZ renderersZ returnersZsdbZ serializersZstatesZthoriumrA z6Bad option for sync_after_install. Defaulting to 'all'r� �< �/� conf_file���)Zmoptszsaltutil.sync_{}�timeout)r z.Synchronized the following dynamic modules: %sz>Failed to create VM %s. Configuration value %s needs to be setr' rN � start_action�Running %s on %sr )"r? r@ r� r> rF r r� r% r& �warningr� rA r[ �gen_keysr( r* Z accept_key�KeyErrorr� r� r, r- rT �deepcopyZDEFAULT_MASTER_OPTSrE Z master_configr� rK �client�get_local_client�cmdr� � exception)rJ r� � local_masterZ sync_sleepr� r� r� r7 rX rO r� �priv�pubZkey_idr� Z master_privr� r Zexpiration_timeZmopts_Z conf_pathr r �excZopt_mapZ action_outr r r r� z s� � � ��� � �� � ���� ���� ������� � �z Cloud.createc C s> |� � }tjj�||�}tjj�||�}|�|� | |d< |S )a Create vm config. :param str name: The name of the vm :param dict main: The main cloud config :param dict provider: The provider config :param dict profile: The profile config :param dict overrides: The vm's config overrides rL )rT r? rA � dictupdaterE )rL �mainr9 r< � overridesZvmr r r � vm_config s zCloud.vm_configc C s� i }|d � d�\}}d�||d �}|| jvr&t�d|d |d |� dS z-tjjj| j| |d d�� | j| d i |��}W d � W |S 1 sLw Y W |S t yu } zt� d |d |d |d |� W Y d}~|S d}~ww ) z Extra actions r9 r: r� r� r� rL Nr� zEFailed to perform %s.%s on %s. Configuration value %s needs to be setr ) rF r r� r% r& r? rA r� r� r r )rJ r� r� r� r7 rX r r r r r� ( s> �� � � ����z Cloud.extrasc C s� || j d vrd�|�}t�|� d|iS i }|si }z(tjj�| j d d��}tjj� |�}W d � n1 s9w Y |sBi }W n t yN i }Y n t yX i }Y nw |du r_i }| � � }| j d | } i } |� � D ])\} } tt| ��} || | D ]}|| | | | |< | | | d< | | | d< q�qp| d �d �\}}| j d | | �� }|d= |D ]�}|| v r�| | d } | | d }d �|| |�}t�|� d|i||< q�| �|||| |�}| j d r�tj| j|fd �}|�� dd�|j�i||< q�z;| �|�||< || �s ddi||< t|�dk�r td��W q�| j �dd�du �r8t|| t ��r8|| � dd� W q� tt f�y^ } zt|�dk�rL� dt |�i||< W Y d}~q�d}~ww |S )zl Parse over the options passed on the command line and determine how to handle them r8 zProfile {} is not defined�Errorr �rNr9 r7 r: r6 z {} already exists under {}:{}rN )r. r Z Provisioningz)VM being provisioned in parallel. PID: {}zFailed to deploy VMr rP F� deploy_kwargs) r> r r% r& r? rA �files�fopenZyamlZ safe_loadr �OSErrorrl rD rG rH rF rT r r# ZProcessr� �start�pidr� r r( ry r� rC r rz )rJ r< r| rx r2 r ZmccZmain_cloud_configr� �profile_detailsr� r� �valZ prov_namer� r� r7 �provider_detailsrL Zdrivr� �processr r r r r{ I s� �� � � � �� � ��zCloud.run_profilec C sj i }i }t |�}| �� �� D ]�\}}|s n�|�� D ]�\}}|s" n�d} d�|| jd �} | | jvr:t�d| � d} |�� D ]�\} } |sF n�| |vrit| t �sQi } d| v r`| d |v r`| d } n t� d| ||� q>| du r�|� | �du r{|� | g i� || � | � q>tjjj| j| d �||g�d ��M ||vr�i ||< ||| vr�i || |< tjjjdi |��}|r�| j| | |dd �|| | | < n| j| | dd �|| | | < |�| � W d � n1 s�w Y q>qqt � } |�r ||d < t � }|�� D ] \}}|�t |��}q�|�|�} | �r t| �|d < t|�|d< |�s"|S | �r'|S t|�|d< t|�|d < |S )zX Perform an action on a VM which may be specific to this cloud provider Tr� r� z)'%s()' is not available. Not actioning...Fr� z-vm:%s in provider:%s is not in name list:'%s'Nr: r� ��callzInvalid Actionsr� zNot Actioned/Not Runningr )r� rl rD r r> r� r% r� ry r� r� r( rE r� r? rA r� r� r* r Z clean_kwargsr� �union� difference�list)rJ r| r r Zinvalid_functionsr� r� r7 r� Zvalid_functionrX r� r� Z missing_vmsZinvalid_func_vms�keyr r r r r� � s� � � �� �� zCloud.do_actionc C s | � |�}t|�dkrtd�|d�dd� |D �����|�� \}}d�||�}|| jvr5td�|||���t�d||� t j j j | j| d �||g�d ��* |rd||| j| d |d �iiW d � S ||| j| d d�iiW d � S 1 szw Y d S )z= Perform a function against a cloud provider r z=More than one results matched '{}'. Please specify one of: {}r� c S s g | ] \}}d � ||��qS )r; �r )r r� r7 r r r r! s z%Cloud.do_function.<locals>.<listcomp>r� zUThe '{}' cloud provider alias, for the '{}' driver, does not define the function '{}'z4Trying to execute '%s' with the following kwargs: %sr: r� �function)r# r Nr" ) r� r� r r r* rC r� r% r� r? rA r� r� )rJ r� r r r� r� r7 rX r r r r� � s6 ��� ����$�zCloud.do_functionc C s< | j d �� �� D ]�\}}|�� D ]r}d�|�}|| jvrIt�d|||||� | j d | �|� || j d vr9q| j d | sH| j d �|� qtj j j | j| d� ||g�d��! | j| � du rtt�d||� | j d | �|� W d � n1 s~w Y q|| j d vr�q | j d | s�| j d �|� q dS ) zV Remove any mis-configured cloud providers from the available listing r6 z{}.get_configured_provideraN The cloud driver, '%s', configured under the '%s' cloud provider alias, could not be loaded. Please check your provider configuration files and ensure all required dependencies are installed for the '%s' driver. In rare cases, this could indicate the '%s()' function could not be found. Removing '%s' from the available providers listr: r� Fz�The cloud driver, '%s', configured under the '%s' cloud provider alias is not properly configured. Removing it from the available providers list.N) r> rT rD r r� r% r rC r? rA r� r� r* )rJ r� r� r7 rX r r r Z __filter_non_working_providers! sJ ���� ����z$Cloud.__filter_non_working_providers�rh F)rh FN)r� �F)Tr� r� ) r� r� r� r� rM r� r� r� r� rl r� r� rf rd r_ r� r� r� r� r� r� � staticmethodr r� r{ r� r� r� r r r r r� � s: + %A �9 " ) ( ( + !V` "r� c @ s` e Zd ZdZdd� Zddd�Zdd d �Zd d � Zd d� Zddd�Z dd� Z ddd�Z dd� Z dS )r\ z3 Create a VM stateful map execution object c C s t �| |� | �� | _d S r� )r� rM �read� rendered_mapr� r r r rM \ s z Map.__init__rh Fc C sz | � � �� }i }|�� D ]�\}}t|�}|| jd vr8d|vr#i |d< d�|d�|��}t�|� ||d |< q | � |||�} | �� D ]B\} } | �� D ]9\} } | �� D ]0\}}| |vr_i || < | || vrki || | < ||| | |< z|� |� W qS t y� Y qSw qKqC|s�q | jd | }|d � d�\} } |D ] }| |vr�i || < | || vr�i || | < d|| | |< q�q |S )Nr8 �ErrorszBNo provider for the mapped '{}' profile was found. Skipped VMS: {}r� r9 r: �Absent) r- rT rD r� r> r r* r% r� r� r� r rF )rJ rn r� r. �interpolated_mapr< Z mapped_vmsr| r2 r� r� r� r7 r� r� r� r r r r r1 ` sR � ��� �zMap.interpolated_mapNc C s� | j |d�}|�� �� D ]B\}}|dkrq |�� �� D ])\}}|�� �� D ]\}}|dkr6|| | �|� q%|| | sD|| �|� q|| sN|�|� q |S )N)rn r/ r0 )r1 rT rD rC ) rJ rn � query_mapr� r� r7 r� r� r� r r r � delete_map� s � � �zMap.delete_mapc C s� | � d�}dgddgddgd�}g }|�� D ]>\}}|�� D ]5\}}|�� D ],\} } z|| } W n ty@ t�d|� t� �w | dkrR| d �� | v rR|�| � q&q q|S ) Nrh �stoppedZrunningZactive)r �stopr� zxThe use of '%s' as an action is not supported in this context. Only 'start', 'stop', and 'reboot' are supported options.r0 �state)r1 rD r r% r& r r� r� ) rJ r� r2 Zmatching_statesZvm_namesr� r� r7 r� r� r� Z state_actionr r r �get_vmnames_by_action� s2 � �� ���zMap.get_vmnames_by_actionc C s d}| j �dd�du rH| j �dd�du rB| j �dd�du r n�| j �d�| j �d�vr6t�d| j d � t� �| j d | j �d� }n�| j d }n�t�| j �}d|d< tj� |�| _t j � | j d �sv| j d � d �svt�d | j d � t� �| j d � d �r�| jjd | j d �}n| j d }z%| j �d d �}tj�| j i �}| j �d�}| j �d�}t|||||�}W n! ty� } ztjd| j d |tjd� i W Y d}~S d}~ww d|v r�tjj|| j d dd�}|s�i S |�� �� D ]�\} } t| ttf��rNi } | D ]H} t| t��r | di} | �� D ]6\} }|du �s t|t ��r!i }z|� d| � W n t �y@ t�d� i Y S w || | < �q�q | || < q�t| t ��roi } | �� D ]\} }|� d| � || | < �qZ| || < q�t| t��rx| g} i || < | D ] } d| i|| | < �q~q�|S )zH Read in the specified map and return the map structure Nr' r} Z map_pillarr= z8The specified map not found in pillar at 'cloud:maps:%s'�localZ file_clientzsalt://z+The specified map file does not exist: '%s'z cp.cache_file�rendererz jinja|yamlZrenderer_blacklistZrenderer_whitelistz)Rendering map %s failed, render error: %sr� ZincludeF)�verboserL z�Cannot use 'name' as a minion id in a cloud map as it is a reserved word. Please change 'name' to a different minion id reference.) r> r( r% r& r rT r r? r� Z MasterMinionr� rK r� r� Z functionsr� Zrenderr r r� r� r@ Zinclude_configrD ry r& �tuplerz �bool� setdefault�AttributeErrorr� )rJ Zmap_Zlocal_minion_optsZ cached_mapr9 ZrendZ blacklistZ whitelistr r< Zmapped�entries�mappingrL r r r r r- � s� � � ���� �� � �zMap.readc C s� |d u r6|d � � D ])}g }z|d }W n ty g }Y nw |D ]}| j|t|�|d�r2 dS q"q dS ||v r<dS |�|� z |d | d }W n tyV g }Y nw |D ]}| j|t|�|d�rh dS qYdS )Nr� �requires)�seenr TF)rU r � _has_loopr& r� )rJ r� rB r rU Zmachines�machiner r r rC / s6 ���� ��z Map._has_loopc C s� z|d }W n t y | Y S w g }|D ]:}z|d | }W n$ t yD z|d | }W n t yA d}t�|� t|��w Y nw |�| �||||�� qt|�d }|S )NrA r� �existingz Missing dependency in cloud mapr )r r% r& r r� �_calcdep�max) rJ r� rD r] �levelZdeplistZlevelsrL r2 r r r rF I s, � ��� z Map._calcdepc s di i}| j |d�� t� }t� }t�| j�}|�� D ]�\}}|| jd vrGd�|d�dd� |D ���}t � |� d|vr@i |d< ||d |< q| jd � |�} |�� D ]�\} } d | v rn| d | d krn| � d �� d �\} } n | � d �� d �\} } t�| jd | | �}|d= t jj�|| � t�|�}d D ]}d �|�}|| v r�t �d|| |� | �|�| |< q�d| v r�d|v r�d| d v r�d|d v r�|d d �| d d � | d d= | d s�| d= t jj�|| �}||d | < |d � d �\} } |�| | | f� qSq� fdd�}� �� D ]\\} }|�� D ]R\} }|�� D ]H\}}|�| | |f� ||d v�r.�q||�}|�s7�q|D ]&}||d v�rD n t �d|� d|v�rSi |d< |d �|�|d |< �q9�q�q�q d| jv �r�| jd �r�| jd du �r~td��|�|�|d< |S )z9 Create a data map of what to execute on r� r� r8 zjThe required profile, '{}', defined in the map does not exist. The defined nodes, {}, will not be created.r� c s s � | ]}d � |�V qdS )z'{}'Nr( )r r� r r r � <genexpr>l s � z Map.map_data.<locals>.<genexpr>�errorsr9 r: r6 )r� r� r� ZvolumesrA zmap_{}z�The use of '%s' on the '%s' mapping has been deprecated. The preferred way now is to just define '%s'. For now, salt-cloud will do the proper thing and convert the deprecated mapping into the preferred one.r� r� c sZ i }� � � D ]$\}}|� � D ]\}}|� � D ]\}}|| kr(||vr(|d ||< qqq|S )Nr6 )rD )rL r� r� r� r7 r� r� r� �r� r r �get_matching_by_name� s ���z*Map.map_data.<locals>.get_matching_by_namez0%r already exists, removing from the create map.rE ZhardZenable_hard_mapsFz�The --hard map can be extremely dangerous to use, and therefore must explicitly be enabled in the main configuration file, by setting 'enable_hard_maps' to Truer� )rl r� rT r r. rD r> r r* r% r& r( rF r? rA r rE r rC r� r r% )rJ r� r ZexistZdefinedr. Z profile_namer� r2 Z profile_data�nodenamer r� r7 r ZnodedataZsettingZ deprecatedrL r� r� rL r� r� �itemr rK r r} ] s� �� � � � 8 � ����z Map.map_datac $ C s| | � |�rd}t�|� t|��|d �� D ]#\}}t�d|� d}| �||||�}t�d||� ||d | d< qz|d �� }W n tyO i �� }Y nw |D ]#\}}t�d|� d}| �||||�}t�d||� ||d | d< qRt |d �� dd � d �}i }| j d r�g } d } d } d } d } �z^t d d� |D ��\} }| } t�d| � t j �d|| j �du r�td��t�d|d � t jj�t j �d|| j ��\}}||d< ||d< t jj�� }t jj�|d�� }|�|� W d � n1 s�w Y t jjj|| j d d�} t�|� |�dd �d u �r=|� d i � d |d v �r'|d d } d |d v�r=d |d d < | d u�r=| |d!< |D ]F\}}t j jd|| j d d"�}|du �rU�q?t�d#|d � t jj�t j �d|| j ��\}}||d$< ||d%< |� d&i � |d& � ||i� �q?d}|d �d'd��r�|d �d d �d u�r�d }| j ||d(�}t |t ��s�t�d)|� nd*|v �r�td+�!|d* ���| j �d,d�d u �r�|�d-i ��p�|�"d-i �}|�d.|�d/d ��} | d u �r�td0�!| ���||| < W n, t#�y t�d1� tj$�%| j d2 d3�}tj$�&|��r t jjj|| j d d�} Y nw | j �'� }| j d �r3t�d4� d|d5< | d u }|D ]�\}}|| | fv �rF�q9d |v �ra|d �d'd��ra|d �d d �d u�rad }| d u�ro|du �ro| |d!< | d u�r�|� d i � |d � d | � | j d �r�| �(||||d6�� �q9z+| j ||d(�||< | j �d,d�du �r�d-|v �r�t || t ��r�|| �"d-d � W �q9 t�y� } ztjd7||t)j*d8� d9t+|�i||< W Y d }~�q9d }~ww |�d:d;�D ] }| �,|�||< �q�| j d �r�| �r�d<| j v �r | j d< } nt-| �} t�d=| � t.t/| | d>�} | j d? �r�g } d@} t0|d �1� dAd � �D ] \}}| �(g � | dB7 } |D ] } | | �(| d � �qA�q2i }| D ]=}!t�dC| j d? dD�%|!�� t j2�3� � }"|� |"j4dE�%|!�| j d? | j dF dG dHdI�� W d � n 1 �s�w Y �qT| D ]}#|t t5|#�6� �� t t5|#�1� ��dJ< |� |#� �q�|S | D ]}#|� |#� �q�|S )Kz4 Execute the contents of the VM map z,Uh-oh, that cloud map has a dependency loop!r� z Calculating dependencies for %sr z Got execution order %s for %srH rE c S s | d d S )Nr rH r ��xr r r �<lambda>� s z Map.run_map.<locals>.<lambda>)r' rN Nc s s, � | ]\}}|� d d�du r||fV qdS )r� FTN)r( )r rL r< r r r rI s � ��z Map.run_map.<locals>.<genexpr>zCreating new master '%s'rO FzTCannot proceed with 'make_master' when salt deployment is disabled(ex: --no-deploy).z Generating master keys for '%s'rL r� r� r� �wZ hash_type)Zsum_type� make_minionTr� r� r� z 127.0.0.1� master_fingerr� r� r� r� Zpreseed_minion_keysr �r z/Master creation details is not a dictionary: %sr/ z?An error occurred while creating the master, not continuing: {}rP r Z salt_host�hostz2Host for new master {} was not found, aborting mapzNo make_master found in mapr� z master.pubziSince parallel deployment is in use, ssh console output is disabled. All ssh output will be logged thoughZdisplay_ssh_output)r> rL r< r � Failed to deploy '%s'. Error: %sr� r r� r r0 zCloud pool size: %sr� r r c S s | d S )NrH r rO r r r rQ � s r r r� rw r r� r& )r Ztgt_typer )7rC r% r& r rD r� rF r� r �sortedr> rG r? r@ r� r rA r[ r r Zmkstempr �write�cryptZ pem_fingerr� �unlinkr( r= rE r� ry r� r rC � StopIterationrK r* r� rT r� r� r� rz r� r� r4 �_create_multiprocessingr rU r r r rH rI )$rJ r� r2 r' r rH Z existing_listZ create_listr� r� Z master_nameZmaster_minion_nameZ master_hostrT Zmaster_profiler r Zmaster_temp_pubZmtprL r< rS r �outr r� r> r r0 r� Z actionlist�grprN �groupr r� r r r r~ � s� � �� ��� � � � � � � � � � �� ��� � � �� ��� �� � �� �� �� $�z Map.run_mapr* r� )NNr+ ) r� r� r� r� rM r1 r3 r7 r- rC rF r} r~ r r r r r\ W s . s r\ c C s t � t jt j� dS )zh Make every worker ignore KeyboarInterrup's since it will be handled by the parent process. N)�signal�SIGINT�SIG_IGNr r r r r� � s r� c C s� t jj�� d| d d< t| d �}z |j| d | d d�}W n( tyG } z tjd| d |t j d � | d d t |�iiW Y d }~S d }~ww | d � d d �d u r]t |t�r]|�dd � | d t jj�|�iS )z� This function will be called from another process when running a map in parallel mode. The result from the create is always a json object. �jsonr> r� r< r rU rW rL r� r NrP Fr )r? rA rZ � reinit_cryptor� r� r r% r& r� r� rz r( ry r� rC r] r^ )r� r r[ r� r r r r �create_multiprocessing� s, �� �� � rf c C s� t jj�� d| d d< t j�| d �}z3|d�| d � }t jjj|d� | d | d g�d�� || d �}W d � n1 sAw Y W n( t yo } z t j d | d |t jd � | d d t|�iiW Y d }~S d }~ww | d t jj�|�iS )z� This function will be called from another process when running a map in parallel mode. The result from the destroy is always a json object. rd r> r� r� r7 r: r� r� rL Nz Failed to destroy %s. Error: %sr� r )r? rA rZ re r� r� r r� r� r* r r% r& r� r� rz r] r^ )r� r r� rX r� r r r r �destroy_multiprocessing s2 �� ��� �� rg c C s� t jj�� t| d �}z?t jjj|j| d d�| d | d g�d�� | d | d t jj � |j| d � �fW d � W S 1 sDw Y W dS t yr } zt j d| d |tjd � | d | d d fW Y d}~S d}~ww ) z` This function will be called from another process when building the providers map. r> rX r: r� r7 r� Nr� r� r )r? rA rZ re r� r� r� r� r* r] r^ r r% r� r� r� )r] r r[ r� r r r � run_parallel_map_providers_query- s, ��(� � ��rh c O � t t�| d i |��S �Nr )r rh �r �kwr r r r� K � r� c O ri rj )r rg rk r r r r� O rm r� c O ri rj )r rf rk r r r r] S rm r] )NNNNNNNr� )5r� rT r� r� r# r� ra r, r � itertoolsr Z salt.clientr? Z salt.configZ salt.loaderZ salt.syspathsZsalt.utils.argsZsalt.utils.cloudZsalt.utils.contextZsalt.utils.cryptZsalt.utils.dataZsalt.utils.dictupdateZsalt.utils.filesZsalt.utils.userZsalt.utils.verifyZsalt.utils.yamlZsalt.exceptionsr r r r Z salt.templater ZCryptodome.RandomZ Cryptodome� ImportErrorZ Crypto.RandomZCryptoZ getLoggerr� r% r r4 r5 r� r\ r� rf rg rh r� r� r] r r r r �<module> s� ��� �d E j
Save