Package Processing

Module Shifting

Created on Sat Dec 12 13:55:47 2020

pycsamt.ff.processing.corr.buildBlock_freqRhoOrPhase(dictRhoOrPhase)

From a dictionnary, build matrix like ndarray (nfreq, Rho|Phase)

Parameters

dictRhoOrPhase – dictionnary of station and values. We assume stations is sorted to first station to the last

Returns

Block matrix of rho or phase

pycsamt.ff.processing.corr.interp_to_reference_freq(freq_array, rho_array, reference_freq, plot=False)

Interpolate frequencies to the reference frequencies.

Parameters

• freq_array (array_like) – frequency array

• reference_freq (float) – frequency at clean data

class pycsamt.ff.processing.corr.shifting(data_fn=None, freq_array=None, res_array=None, phase_array=None, verbose=0, **kwargs)

processing classshifting processing workflow correction class

deal with AVG Zonge station file “*.stn” or SEG-EDI file.

data_fnstr

path to Zonge *AVG file or SEG-EDI files or jfiles

res_arrayarray_like (ndarray,1)

apparent resistivities uncorrected data

freq_arrayarray_like (ndarray,1)

frequency array during survey

phase_arrayarray_like(ndarray,1)

phase array during survey

More attribute will populate :

Attributes	Type	Explanation
_rj	array_like	static shift factor
rho_static	array_like	corrected data from files
_reference_freq	float	reference value to start corrected data . If provided , function will use for data correction.if notprovided, program will search the reference frequency automatically and set it for computation.
verbose	int	control the level of verbosity. Higher more messages.

Methods	Description
TMA	Trimming Moving Average computation.
FLMA	Fixed Length Dipole Moving average computation.
AMA	Adaptative Moving average . see Biblio.

More attributes can be added by inputing a key word dictionary

Note

Reference frequency doesnt need to be on frequency range If value is not in frequency array, program will interpolate value

Example

>>> from pycsamt.ff.processing.corr import Shifting
>>> path =  os.path.join(os.environ["pyCSAMT"],
...                       'pycsamt','data', LCS01.AVG)
... static_cor =shifting().TMA (data_fn=path,
...                                 reference_freq=1024.,
...                                 number_of_TMA_points =5 )
... print(static_cor)

AMA(data_fn=None, dipole_length=50.0, number_of_skin_depth=3.0, freq_array=None, reference_freq=None, **kwargs)

Adapative moving average AMA to correct apparent resistivities. AMA filter estimates static-corrected apparent resistivities at a single reference frequency by calculating a profile of average

impedances along the length of the line. Sounding curves are then

shifted so that they intersect the averaged profile. The highest frequency with clean data should be selected as the static-correction reference frequency.

Parameters

number_of_skin_depths – skin depth for filter length

:type number_of_skin_depths : int, float, default is 3.

Returns

resistivities corrected with ama filter

:rtype:dict ,

See also

for other parameters explanations see the docstring of FlMA

1. read from edipath or jpath

Example

>>> from pycsamt.ff.processing.corr import shifting
>>> edipath =os.path.join(os.environ['pyCSAMT'], 'data', 'edi')
>>> corr_obj= shifting(data_fn =edipath)
>>> corrapp  = corr_obj.AMA(number_of_points=7,
...                             reference_freq=1024.)
>>> corr_obj._rj
>>> corrapp

2. read Zonge avg file

Example

>>> from pycsamt.ff.processing.corr import shifting
>>> avg_path = os.path.join(os.environ['pyCSAMT'], 'data', 'avg')
>>>  csamt_obj =CSAMT(data_fn =os.path.join(avg_path, 'K2.AVG'),
...                  profile_fn=os.path.join(avg_path, 'K2.stn'))
>>> corr_obj= shifting(data_fn =os.path.join(avg_path, 'K2.AVG'),
...                     profile_fn=os.path.join(avg_path, 'K2.stn'))
>>> corrapp  = corr_obj.AMA(number_of_points=5, reference_freq=8000.)
>>> print(corr_obj._rj)
>>> print(corrapp)

FLMA(data_fn=None, dipole_length=50.0, number_of_dipole=5.0, reference_freq=None, **kwargs)

Fixed length-dipole moving average FLMA to correct apparent resistivities.The FLMA filter estimates static-corrected apparent resistivities at a single reference frequency by calculating a profile of average impedances along the length of the line. Sounding curves are then shifted so that they intersect the averaged profile. The highest frequency with clean data should be selected as the static-correction reference frequency.

Parameters

• data_fn (str) – full path to data file , could be[AVG|EDI|J]

• dipole_length (float, default is 50.) – value of dipole length in meters

• number_of_dipole – number of dipole to cover hanning window width

:type number_of_dipole : int, float, default is 5.

Parameters

• reference_freq (int, float) – reference frequency at clean data , if not provided reference will be compute automaticcally

• freq_array (array_like) – array of survey frequency of the field

• res_dict_arrays (ndarray_array, dict , optional) – resistivity array of data collected on the field, could be a ndarray of resistivites at each frequency of each sites or a dictionnary of array

• phase_dict_arrays (ndarray, optional) – phase values of data at each stations could be an ndarray of phase with ndarray(len(freq), len(nstaions)) or dictionnary of phase values at each stations.

Returns

rho corrected at each survey sites

Type

dict

Note

skin depth param is not to used for FLMA filter application. It is not necessary to provide when apply for FLMA filter.

1. read from edipath or jpath

Example

>>> from pycsamt.ff.processing.corr import shifting
>>> edipath =os.path.join(os.environ['pyCSAMT'], 'data', 'edi')
>>> corr_obj= shifting(data_fn =edipath)
>>> corrapp  = corr_obj.FLMA(number_of_dipole=7,
...                             reference_freq=1024.)
>>> corr_obj._rj
>>> corrapp

2. read Zonge avg file

Example

>>> from pycsamt.ff.processing.corr import shifting
>>> avg_path = os.path.join(os.environ['pyCSAMT'], 'data', 'avg')
>>>  csamt_obj =CSAMT(data_fn =os.path.join(avg_path, 'K2.AVG'),
...                  profile_fn=os.path.join(avg_path, 'K2.stn'))
>>> corr_obj= shifting(data_fn =os.path.join(avg_path, 'K2.AVG'),
...                     profile_fn=os.path.join(avg_path, 'K2.stn'))
>>> corrapp  = corr_obj.FLMA(number_of_dipole=7, reference_freq=8000.)
>>> print(corr_obj._rj)
>>> print(corrapp)

TMA(data_fn=None, reference_freq=None, number_of_TMA_points=5.0, **kwargs)

Corrected apparent resistivities with Trimmed-moving-average filter(TMA) .TMA estimates average apparent resistivities at a single static-correction-reference frequency. if reference frequency is not provided , will find automatically.

Parameters

• data_fn (*) – path to avg file or edi file .

• freq_array (*) –

  frequency array of at normalization frequency (reference value)of all stations.

  station j to n .( units = Hz )

• res_array (*) – dict of array of app.resistivity at reffreq. from station j to n.

• phase_array (*) –

  of phase at reffreq.from station j to n.

  (unit=rad)value of frequency with clean data . (unit=Hz)

• stnVSrho_loc (*) – set of dictionnary of all app.resistivity data from station j to n . (optional)

• num_of_TMA_point (*) – window to apply filter .

Returns

rho_corrected , value corrected with TMA filter from station j to n.

Return type

dict

1. corrected data from [AVG]

Example

>>> from pycsamt.ff.processing.corr import Shifting
>>> path =  os.path.join(os.environ["pyCSAMT"],
...         'pycsamt','data', LCS01.AVG)
... static_cor =shifting().TMA (data_fn=path,
...                            reference_freq=1024.,
                            number_of_TMA_points =5 )

2. corrected from edifiles [EDI]

Example

>>> from pycsamt.ff.core.cs import CSAMT
>>> from pycsamt.ff.processing.corr import Shifting
>>> edipath = r'C:/Users\Administrator\Desktop      est\edirewrite'
>>> csamt_obj =CSAMT(edipath =edipath)
>>> static_cor =shifting().TMA( reference_freq =256. ,
...                               freq_array = csamt_obj.freq ,
...                               res_array = csamt_obj.resistivity ,
...                            phase_array =csamt_obj.phase ,
...                            number_of_TMA_points=5)
... print(static_cor)

compute_fixed_and_adaptative_moving_average(filterfunc, data_fn=None, profile_fn=None, dipole_length=50.0, reference_freq=None, **kwargs)

Can use this fonction to compute at the same time FLMA and AMA by setting only the filter func argument . If the function is used set the param filterfunc to the filter we need. Avoid repetition in the code Later.

Parameters

• filterfunc (obj) – filter fonction , can be filter-AMA or filter-FLMA

• data_fn (str) – full path to data file , could be[AVG|EDI|J]

• dipole_length (float, default is 50.) – value of dipole length in meters

• number_of_skin_depths – skin depth for filter length

:type number_of_skin_depths : int, float, default is 5.

Parameters

reference_freq (int, float) – reference frequency at clean data , if not provided reference will be compute automaticcally

1. compute fixed length dipole moving average FLMA

Example

>>> from pycsamt.ff.processing.corr import shifting
>>> edipath =os.path.join(os.environ['pyCSAMT'], 'data', 'edi')
>>> corr_obj= shifting(data_fn =edipath)
>>> res_flma_obj  = corr_obj.compute_fixed_and_adaptative_moving_average(
    filterfunc=Zcc.compute_FLMA,  number_of_points=7,
...                             reference_freq=8192.)
>>> corr_obj._rj
>>> res_flma_obj

2. compute adaptative moving-average AMA

Example

>>> from pycsamt.ff.processing.corr import shifting
>>> edipath =os.path.join(os.environ['pyCSAMT'], 'data', 'edi')
>>> corr_obj= shifting(data_fn =edipath)
>>> res_ama_obj   = corr_obj.compute_fixed_and_adaptative_moving_average(
    filterfunc=Zcc.compute_FLMA,  number_of_skin_depth=7,
...                             reference_freq=8192.)
>>> corr_obj._rj
>>> res_ama_obj

correct_edi(data_fn=None, reference_frequency=None, FILTER='tma', **kwargs)

Method to correct edifiles applying filters.

Availbale filters are tma (triming moving average), flma fixed-length dipole moving average and ama adaptative moving average for Electromagnetic Array Profilin(EMAP). To apply filter for MT data, set FILTER arguments to ss for static shift removal and dist for distortion removal. It’s possible to apply EMPA filters to MT data by setting datatype argument to “mt”.

Parameters

• data_fn (str) – full path to edifiles

• FILTER (str, default is`tma`) – type of filter to write, can be tma or flma for EMAP data or ss (remove static schift) or dist (remove distortion)for MT data . Default is tma assume data provided are EMAP data.

Reference_frequency

refrequency at clean data, optional when apply for filter ss and dist.

Holds others parameters:

Params	Type	Description
filename	str	name of output edifiles
number_of_points	int	weighted window. Default is 7. if one edifile is provided , change weigthed point to 1.
dipole_length	float	length of dipole. Default is 50.m
reduce_res_factor_x	float	static shift factor to be applied to x components (ie z[:, 0, :]).This is assumed to be in resistivity scale
reduce_res_factor_y	float	static shift factor to be applied to y components (ie z[:, 1, :]). This is assumed to be in resistivity scale
distortion_tensor	ndarray	real distortion tensor as a 2x2, np.ndarray(2, 2, dtype=real)
distortion_err_tensor	ndarray	real distortion tensor error as a 2x2 np.ndarray(2, 2, dtype=real)
datatype	str	type of data provided . Could be mt or emap. Detect automatically if None.
savepath	str	full path to save outputfile.

1. corrected single edifile

Example

>>> from pycsamt.ff.processing import shifting
>>>  edifile =os.path.join(os.environ['pyCSAMT'], 'data',
...                           'edi', 'new_csa000.edi' )
>>> corr_obj= shifting()
>>> corr_obj.write_corrected_edi(data_fn = edifile,
...                                 number_of_points =1.,
...                             dipole_length =50., FILTER='ss')

2. corrected multi edifiles EMAP

Example

>>> from pycsamt.ff.processing import shifting
>>>  edipath =os.path.join(os.environ['pyCSAMT'], 'data', 'edi')
>>> corr_obj= shifting()
>>> corr_obj.write_corrected_edi(data_fn = edipath,
...                             number_of_points =7.,
...                             dipole_length =50.,
...                                FILTER='flma')

read_processing_file(data_fn=None, profile_fn=None, reference_freq=None, **kwargs)

Mehod read processing files and load attributes for use.

Parameters

• data_fn (str) – full path to data file , coulb be avg of Zonge International Engineering or edi of Society of Exploration Geophysics or j|dat format of AG.Jones MT.

• profile_fn (str) – full path to station profile file. It’s compulsory to provide this file when read avgfile.

write_corrected_edi(data_fn=None, reference_frequency=None, FILTER='tma', **kwargs)

Method to correct edifiles applying filters.

Availbale filters are tma (triming moving average), flma fixed-length dipole moving average and ama adaptative moving average for Electromagnetic Array Profilin(EMAP). To apply filter for MT data, set FILTER arguments to ss for static shift removal and dist for distortion removal. It’s possible to apply EMPA filters to MT data by setting datatype argument to “mt”.

Parameters

• data_fn (str) – full path to edifiles

• FILTER (str, default is`tma`) – type of filter to write, can be tma or flma for EMAP data or ss (remove static schift) or dist (remove distortion)for MT data . Default is tma assume data provided are EMAP data.

Reference_frequency

refrequency at clean data, optional when apply for filter ss and dist.

Holds others parameters:

Params	Type	Description
filename	str	name of output edifiles
number_of_points	int	weighted window. Default is 7. if one edifile is provided , change weigthed point to 1.
dipole_length	float	length of dipole. Default is 50.m
reduce_res_factor_x	float	static shift factor to be applied to x components (ie z[:, 0, :]).This is assumed to be in resistivity scale
reduce_res_factor_y	float	static shift factor to be applied to y components (ie z[:, 1, :]). This is assumed to be in resistivity scale
distortion_tensor	ndarray	real distortion tensor as a 2x2, np.ndarray(2, 2, dtype=real)
distortion_err_tensor	ndarray	real distortion tensor error as a 2x2 np.ndarray(2, 2, dtype=real)
datatype	str	type of data provided . Could be mt or emap. Detect automatically if None.
savepath	str	full path to save outputfile.

1. corrected single edifile

Example

>>> from pycsamt.ff.processing import shifting
>>>  edifile =os.path.join(os.environ['pyCSAMT'], 'data',
...                           'edi', 'new_csa000.edi' )
>>> corr_obj= shifting()
>>> corr_obj.write_corrected_edi(data_fn = edifile,
...                                 number_of_points =1.,
...                             dipole_length =50., FILTER='ss')

2. corrected multi edifiles EMAP

Example

>>> from pycsamt.ff.processing import shifting
>>>  edipath =os.path.join(os.environ['pyCSAMT'], 'data', 'edi')
>>> corr_obj= shifting()
>>> corr_obj.write_corrected_edi(data_fn = edipath,
...                             number_of_points =7.,
...                             dipole_length =50.,
...                                FILTER='flma')