Fourier transforms and spectra

[fjebaker]

Given an event list or light curve, we want to be able to compute the Fourier transform and obtain the spectra. Specifically, we want the AveragedPowerspectrum.

Much of this code already exists in the Stringray.jl implementation, but need to be cleaned up / rewritten for the new primitives we are working with.

We minimally need something like the from_events constructor to construct an averaged power spectrum from the EventList. This constructor does not need all of the Python functionality, e.g. GTI treatment, as that will be handled separately for now.

We want something like:

struct AveragedPowerspectrum{T}
    freqs::Vector{T}
    power::Vector{T}
    power_errors::Vector{T}
end

I don't think any of the other attributes are relevant for now!

[fjebaker]

Note: this is not a small task. It will involve implementing something like this Python function, which has already been implemented in Julia, but needs revisiting.

The temptation here is to write a lot of code to handle all of the different normalisations. I would urge you to avoid doing that, and instead provide the means by which we can add the other normalisations / features later, but provide for now just the default cases. This will keep the code changes short and manageable whilst we get the core of the functions right, and in doing so avoid having to rewrite a whole load of cases whilst we refine the solution.

[JawadAhmadCS]

hi @fjebaker It would be appreciated if you could provide a bit of the workflow to get started!. I’m thinking of working on this task. after successfully approval of #28

[kashish2210]

@fjebaker This going to my workflow suggest necessary things if @JawadAhmadCS is ohkay with this then assign this to me

my workflow

[fjebaker]

Hi @kashish2210 your notebook looks like a good start! Some comments:

• There is no need in Julia to have functions that are called X_from_Y, since Julia can dispatch the right function from the types. It would be preferred to have a single function called e.g. powerspectrum() which has a number of different dispatches for different types.
• The get_flux_iterable_from_segments relies on a lot of deeply nested code. From what I can tell, all it is doing is generating segments of contiguous counts which are in the GTI. I think it's easier to handle the GTIs seperately for now and to keep the implementation simple. If you could modify your code to just assume the entire EventList or LightCurve is already a single contiguous GTI, then it will make reasoning about generating the powerspectrum much simpler.

Ideally I want to drop the DataFrames and ResumableFunctions dependencies at some point, so would prefer to rely on code that doesn't make use of them. DataFrames, in my experience, makes it a little troublesome to write functions that are easily type-inferred, and resumable functions are not really needed. They are fantastic for co-routines or where you have blocking operations, but ultimately we only need to do fast data processing.

[kashish2210]

yeah I was testing the test cases of the codes in jullia nightly and found out that resumable function doesn't not support there was an error I have also asked in the community
got it will try to implement it according to ur review :

[JawadAhmadCS]

@fjebaker This going to my workflow suggest necessary things if @JawadAhmadCS is ohkay with this then assign this to me

my workflow

No problem, @kashish2210 . If you're more interested, then you can work on. I've done a deep analysis of the Fourier.jl code who already in stingray.jl and understood its workflow and after full understanding to perform it, I created a flowchart, which is attached here. You can check it out, it should make things easier for you.

1:

2:

[kashish2210]

Hi @kashish2210 your notebook looks like a good start! Some comments:

• There is no need in Julia to have functions that are called X_from_Y, since Julia can dispatch the right function from the types. It would be preferred to have a single function called e.g. powerspectrum() which has a number of different dispatches for different types.
• The get_flux_iterable_from_segments relies on a lot of deeply nested code. From what I can tell, all it is doing is generating segments of contiguous counts which are in the GTI. I think it's easier to handle the GTIs seperately for now and to keep the implementation simple. If you could modify your code to just assume the entire EventList or LightCurve is already a single contiguous GTI, then it will make reasoning about generating the powerspectrum much simpler.

Ideally I want to drop the DataFrames and ResumableFunctions dependencies at some point, so would prefer to rely on code that doesn't make use of them. DataFrames, in my experience, makes it a little troublesome to write functions that are easily type-inferred, and resumable functions are not really needed. They are fantastic for co-routines or where you have blocking operations, but ultimately we only need to do fast data processing.

I have started pre-initialization according to ur suggestion :
notebook
if I am in correct direction let me know:)

[kashish2210]

Here @fjebaker as per ur suggestion in #36

struct EventList{T}
    times::Vector{T}
    energies::Vector{T}
end

# Define LightCurve struct
struct LightCurve{T}
    timebins::Vector{T}
    counts::Vector{T}
    count_error::Vector{T}
    err_method::Symbol
end

# Define AveragedPowerspectrum struct
struct AveragedPowerspectrum{T<:AbstractFloat}
    freqs::Vector{T}
    power::Vector{T}
    power_err::Vector{T}
end

and

#constructor 
function powerspectrum(events::EventList{T}, 
                       dt::Real, 
                       segment_size::Real;
                       norm::String="frac",
                       use_common_mean::Bool=true) where T<:AbstractFloat

Check whether I am now in the correct direction or not
Later it can be upgraded for filename and metadata::DictMetadata

[kashish2210]

Hey @fjebaker, I have tried to resolve this issue in my same Pr #36 cause eventually it was needed to be resolved with issue #34
please review and and give your suggestion

[fjebaker]

Hi @kashish2210 please could you seperate this from fixing #34 and open a new PR for it. I don't have the time to review large PRs at the moment, so please keep your changes small and simple 👍