okay it is of course actually a different situatoin from that i'm trying to extract only part of the data, which has gotten all mutated. but now i remember, i found the problem was similar to a different problem, of simply having extra frequencies. so this solves the extra frequencies issue, i believe. maybe i should just implement that, and see how it works. 1729 things are complicated a little because i implemented the rfft thing it makes sense to convert the data to a normal complex fft before processing it, so i can compare with what i just went through 1752 i started making things the same between what works and what doesn't. a difference i ran into is that the starting angles of the broken code start much larger? i think this basically means it's not evenly spaced around a circle: but it is symmetrical about 0 degrees. i don't know whether that works, i guess i'll look at it. this would be so much easier if i could reasonably imagine these things without my issues i guess i imagine very slowly now looks like my frequency generation is missing frequencies! 1757 1803 well i'm narrowing in on the issue. presently it's generating frequencies starting at a large, rather than small, value, and the resulting matrix is not symmetric. understanding why is again difficult for me. ~ to help me figure this out, i'm trying out generating more frequencies than needed. but it's still only generating frequencies for the passed band, and now that there are more it starts off late.
np.angle(np.exp(np.outer(np.arange(4), 2j * np.pi * np.array([0,.25,-.5,-.25])))) * 180 // np.pi array([[ 0., 0., -0., -0.], [ 0., 90., -181., -91.], [ 0., 180., 0., -181.], [ 0., -91., -180., 90.]]) np.angle(np.exp(np.outer(np.arange(4), 2j * np.pi * np.array([0,.75,-.5,-.75])))) * 180 // np.pi array([[ 0., 0., -0., -0.], [ 0., -91., -181., 90.], [ 0., 179., 0., -180.], [ 0., 89., -180., -90.]])
it's making matrices like the second, rather than the first, where one half is off from the other half. it uses this to decide what frequency to start at: min_k = tN / max_period # max_period == tN start_freq = min_k / tN meanwhile the number of frequencies is fN > tN . so it ends up starting with a short period. i'd like to understand why that makes an issue. 1806. ... and see what options are available for focusing the frequencies on a band. if that have to be that dense across the entire scale, so be it. when we generate the matrix, it considers all 4 frequencies, and all 4 offsets. the offsets go across an entire period (2pi), so if the frequencies are not arrange right with each other, some of the resulting timepoints are disaligned. 1808 each frequency is scaled by all the offsets, and that's how the matrix is made. do we need all these offsets? the matrix could be symmetric by design. it looks like it just needs a unique set of angles that balance each other. i wonder. i tried with simply symmetrical angles, and it didn't work. i tried with angles that were even multiples but didn't go around a circle, and it didn't work. i'm a little curious as to why this is, but maybe for now it's good enough to know the angles must go around a circle. i'm looking at N=4 . the angles here are -90, 0, 90, and 180. the frequencies are -2/4, -1/4, 0/4, 1/4 . so, how do these frequencies make angles that extend evenly around a circle? there are 4 scales of frequencies, and 4 total lengths ... each frequency is multiplied by each offset. so 0 goes by 0, 1, 2, 3 -> all get 0 1/4 goes by 0, 1, 2, 3 -> this extends around a circle 1/2 goes by 0, 1, 2, 3 -> this engages opposite ends of a circle twice -1/4 goes by 0, 1, 2, 3 -> this extends around a circle in the opposite order the opposite order produced by -1/4, opposing +1/4, and bounded by 1/2 and 0, makes the symmetry around a circle in every direction in the resulting matrix. the negative and positive frequencies need to evenly distributed between 0 and 1/2, so that the matrix inverts itself when summed the other way. 1823 so if i want a bandpass, i need to consider the entire matrix, that extends around a whole circle but i can possibly assume that the parts that are outside the band are not needed this is something people know well already. you can take the fft, and set output portions to 0, and then take the ifft, and you get a band pass (i think! i dunno). so for the start of making it make sense, i would be making a much larger frequency matrix. and basically i propagate all the zeros in properly if i want to make it smaller. maybe! i might be missing something. so if max_period is 8, and there are 14 frequencies, what does that look like or mean? it's confusing. it's easier if max_period is 7 and there are 14 frequencies. then they can be evenly distributed. it's of course possibly fine to have 16 frequencies. i think i'm missing a couple understanding here, to figure out how to move forward with possible appropriateness. - the frequencies work best if they are evenly distributed between -1/2 and 1/2 . - the signal, in theory, contains no information above a set frequency - the set frequency is not a multiple or factor of the data length - the data length and the frequency count do not need to be related. the frequencies just need to be balanced. so, there are a lot of low frequencies that can get included when done normally. I'm thinking a little about low frequencies, and how things walk angles. i wonder if the angle walking can sort out with them. if i consider the waveform in the frequency domain, then it's made of a bunch of complex-valued things. when it's viewed in the recording, it's as if it's frequency shifted. all the complex-valued things sit at different frequencies than they did before. their indices in an fft are slid up or down. but they are still at the same angles. when the fft or ifft act, they multiply everything by their angle matrix, rotating all the values and recombining them. the matrix needs to have a positive angle that balances every negative angle, summing it needs to work a certain way. [it looks like the angles need to form an even circle too, which i haven't understood why] but it seems like there would be some way to consider this, and account for the presence or lack of information. i think of these things so incredibly slowly ... so what happens if we just have a bunch of evenly distributed frequencies? how does that work for finding one inside another? we'd want all the frequencies of the waveform so our N gets squeezed very small then we want to cover the whole circle of frequencies originally the distance between frequencies is 1/N but now we want it to be max_period_ratio / N . so i think it's doable. i'd consider the frequency circle as being scaled from N to N * max_period ... or something like that. i'd break it into enough tiny chunks such that size is right then i'd extract only the region of interest, assuming the transform is zero elsewhere. so what is this N * max_period thing. what should it actually be. the frequencies are distributed between +-1/2 over (N+1) using N there is appropriate if max_period == N. if max_period < N, then what? we possibly need more frequencies. say max_period is N/2 . then we imagine N*2 frequencies, instead of N. new_n = old_n * 2 # max_period = old_n / 2 it looks like new_n = old_n / (max_period / old_n) = old_n ** 2 / max_period say max_period is old_N/3, then we imagine N*3 frequencies new_N = old_N**2 / max_period = old_N**2 / (old_N / 3) = old_N * 3 . ok i have fN = tN * tN / max_period what happens if max_period is very near tN? then fN = tN * tN / (tN * 0.99) fN = tN / 0.99 it just scales up fN a little bit. are the periods the right length, in the imagined extended time sequence? if max_period = tN * 0.99, then the space between samples to recover each waveform sample is ... what ... uh ... there are N samples within max_period, and the data is N samples long. max_period / N is the amount of space, in recording samples, we want between time points. it's <1, as it should be. so that's tN * 0.99 / tN = 0.99 and then fN = tN / 0.99 so fN * 0.99 = tN it works out. the right distance is between the samples, because there are tN samples in a waveform length of data. ok, so basically i generate the same frequencies, but i generate many more, according to , um ... fN = tN * tN / max_period . 1842 1853 it's notable that filtering in the frequency domain does not equate to leaving frequencies out, at least because they still contribute to the sum that things are scaled by in the inverse 1858 i found an inaccuracy in the approach. with an arbitrary scale, the number of frequencies can become noninteger. this may mean my test data will fail unless i ensure it is an integer. 1859 it's processing the basic normal fft data accurately now that it wasn't succeeding at before. i'm not filtering the fft, just using the whole thing. 1901 it's still not reconstructing the smaller wave right. i think the loop for checking is much tighter now, though. the points at which the wave is sampled should be the same distance apart. 1904 looks like it's still generating freqs in a circle when a max_period is specified. 1908 the reconstructed waveform has samples equal in value to ones from original waveform again. it's still set to the simple N/2 period. when i set the output size to the length retained in that waveform (N/2), it outputs it exactly correctly. so that's progress. i'm not expected as much from the variant that is offset by half a sample. ok, max_period = 3.73333 and tN = 8 . the frequencies aren't exact, tN * tN / max_period is nonintegral. the max_period is ideally a factor of the square of the input length. is that doable? 3.73333 = 8 * (8 - 1) / (8 * 2 - 1): 8 *7/15 uhhh i could just scale up the input length so there's a 7 factor in there. wonder if it gets too big. oh, the 7 is a function of th einput length ;p 1917 i doubled the recording length, which happened to meet the factor-of-square restraint. i forgot to increase the output size, so it was still squeezing it into half length. the "reconstructed" output was exactly equivalent to the first half of the wave. the second half is nowhere to be seen. maybe an artefact of the wavelets-of-self situation. also an artefact of making max_period this value ... it ends up not scaling the waveform at all! it's pasted right in :S 1922 i tried out removing the assertion that the period be a fraction of the total length, and when it isn't it's doing the same old thing of having the first and last sample match but not the ones in the middle. if the period is a fraction of the total length, how do we use moireness to get other parts of the signal? it will just divide evenly ! somehow it actually recovers the signal under those conditions, which is strange. oh this makes sense because it is not downsampled in the condition i tested. hum ......... 1927 this part seems difficult. - it samples the signal evenly in the time scale of the fake recording - it does this via a fourier transform, so atm it is setting a frequency for every combination of waveform offset and recording offset, but a filter can seriously reduce those. it eases confused analysis. - there must be some situation where the signal slides along under the recording, and its parts can be picked up. it could slide along in some kind of even multiple - this kind of could mean a lot of subsamples. for example if the waveform is wN long and the recording is cN = wN * 2 + 1 or cN = wN * 2 - 1, then there is some kind of disagreement between the two. say the waveform and recording are both N long, but the max period is 1/(N+1) or 1/(N-1) whatever is needed so that the a single sample is made of every period, and each one is offset in the wave by 1/N . so the subwave samples in the larger sequence have a frequency of like 1/N^2 or something? max_period = 1/N .. o roff a little bit ok so if on the 2nd sample, i want the 2nd sample of the 2nd waveform to pop through, then i would need to advance along the waveform by N + 1. so 1 unit in the larger recording, is N + 1 for the waveform. then the period size is .... i'm not sure. a period is N. 1 is N + 1. so period length is 1 / (N + 1) * N = N / N + 1 . 1934 ok if i am starting with period length, then i'll need to form the sampling indices based on that. maybe i can just calculate them. ha, this is ridiculous: i directly sample the wave onto the recording. it looks identical. the system does figure out the max_period of 0.8888888 and produce the right output though. ok, so now something that makes the data get shuffled around. some amount that it is advanced every waveform can i maybe make it shorter ... N / (N - 1) ? so it progresses backward? this makes sense to try because it means making that tiny period larger. now max_period is 1.142857142857 unfortunately, the output is wrong. i'm thinking when i sample it i need to use the same frequency count, or it's a different wave being interpolated from and sampled from. 1939 no, that doesn't seem to be it. blarg dangit. i should probably low-pass the waveform and see if it comes out similar to what it was before. okay, it's still not looking good with lowpassing, and that makes sense, because i still have all the low frequencies in there and it's only sampling a single period, so it gets all the other artefacts overlayed. 2008 ha i'm actually ignoring the upsample flag entirely now. the code was short circuited when i switched to the fN approach. my goodness. 2014 sadly it's not working. a gain is that i built further thoughts around FFT. it's quite hard to engage my cognitive issues while experiencing strong pressure to stop working on something. i think i can improve at this.