https://cmb-s4.uchicago.edu/wiki/api.php?action=feedcontributions&user=Michaelray&feedformat=atomCMB-S4 wiki - User contributions [en]2022-08-13T04:01:44ZUser contributionsMediaWiki 1.34.2https://cmb-s4.uchicago.edu/wiki/index.php?title=PureB_by_Messenger_Method&diff=9243PureB by Messenger Method2019-09-12T15:23:41Z<p>Michaelray: </p>
<hr />
<div>''Michael Ray, 2019-09-xx''<br />
----<br />
<br />
Dr. Bischoff and myself have been working on a new approach to a pure-B estimator for CMB polarization data. The method was originally proposed in 2012 (Elsner & Wandelt 2012) and has been dubbed the "messenger method" approach to weiner filtering. To make a pure-B weiner filter, we represent the noise covariance matrix as the noise covariance plus the E component of the signal covariance (Bunn & Wandelt 2017), so that the filter treats any E modes as noise. This means the weiner filter equation becomes:<br />
<br />
data<sub>wf</sub> = (S<sub>b</sub>/(S<sub>b</sub>+S<sub>e</sub>+N))*data<br />
<br />
The difficulty in weiner filtering for CMB analysis lies in the fact that we can only easily write down S in the sphereical harmonic basis, and N in the pixel basis. In fact, S is diagonal in spherical harmonic space, and N is diagonal in pixel space. Because of this, we must use a work around in order to weiner filter maps. The general approach of the messenger method is to introduce a new field, t, which is called the "messenger field". This field has covariance T, which is proportional to the identity matrix and represents the homogeneous portion of the noise covariance. Being proportional to the identity matrix, T can be written diagonally in any basis we choose (Elsner and Wandelt 2012).<br />
<br />
Since we have multiple matrices that can only be written in one basis, this suggests some sort of iterative algorithm to solve for a weiner filtered map where we use the messenger field t and its covariance matrix to mediate between pixel and sphereical harmonic space. Elsner and Wandelt lay out this procedure in their paper and they come up with two equations to be solved iteratively for a weiner filtered map. In their equations, there is a lambda parameter which is a scale factor used to artificially increase the covariance T. This is useful in speeding up convergence of the algorithm. At the end of the algorithm, lambda must be reduced to 1 in order to solve for the true weiner filtered map.<br />
<br />
In our implementation, lambda begins at a value of 1300, then on the second iteration it is decreased to 100, and from then on out, lambda -> lambda*eta on each iteration, where eta = 0.825. When lambda reaches 1, our algorithm iterates another 5 times. We have experimented fairly extensively with other cooling schedules for lambda and found that our method gives the best convergence in the least amount of time. We are able to filter a single set of Q and U maps in 34 minutes on a single processor. Shown below is an example of a filtered and unfiltered map for visualization of what our algorithm is doing. The input maps below came from CMBS4 combined simulations at 95 GHz using experiment configuration 04.00. So far, the only maps we have done any work with are at 95 GHz and in this experiment configuration. The algorithm is set up to work with any set of Q and U maps, however, and thus can easily be used at any frequency with any experimental configuration.<br />
<br />
<br />
[[File:Q input map.png|frame|Figure 1: Input Q signal plus noise map. This is zoomed in on the unmasked region of the map.|center]]<br />
[[File:Q output map.png|frame|Figure 2: Output Q signal plus noise map. This has been filtered through our pure B estimator. This map is also zoomed in on the unmasked region although the algorithm extends the map into the unmasked region.|center]]<br />
<br />
<br />
The algorithm takes in a set of two maps, corresponding to a Q and U measurement, and outputs a B only weiner filtered set of Q and U maps. One can visually see that although the input map has a defined edge, the algorithm extends the output map into the unobserved region. This is because the code is coming up with a full sky weiner filtered map and thus any power detected at low ell will result in map fluctuations at large angular scales. It is also visually apparent that the input Q map contains mostly E modes (fluctuations that appear straight up and down or straight across), while the output Q map looks like it's all B modes (diagonal fluctuations). This is exactly what we would hope to see from a pure-B estimator. Note that there is a scale difference between the two plots. This is because the input map contains E signal, so the Q map will have a larger amplitude in each pixel for the input map than the output map which contains only B modes.<br />
<br />
There is still some noise bias in the output map, however we can remove that through using monte carlo methods. There is also a suppression factor involved in the output data which is the reason for the scale difference in plots. This factor is corrected for through the use of band power window functions. Both the noise debiasing and suppression factor correction are made at the power spectrum level only. We are not capable of applying noise and suppression factor corrections to maps.<br />
<br />
Shown below is a plot of 100 spectra which are cleaned signal plus noise maps at 95 GHz taken from the 04.00 CMBS4 experiment configuration. The bins used in this analysis began at ell of 20, and went up through ell of 370 with a bin size of 35. So, bin number zero is ell of 20 - 55. Bin 9 (last bin) is ell of 335 - 370. Also included is the mean of these 100 spectra and the expectation value which was calculated using band power window functions and theory BB spectra.<br />
<br />
<br />
[[File:Filtered specs mean vs expectation.png|frame|Figure 3: Band powers of 100 filtered signal plus noise simulations. These were filtered using the messenger method estimator. Also shown is the mean and the expectation value calculated using band power window functions and BB theory spectra.|center]]<br />
<br />
<br />
As one can see, the expectation value for our estimator runs directly through the middle of our simulations and is very close to our mean. It is worth noting that there is definitely something out of the ordinary happening in our last bin. In this bin, we routinely get results that do not agree with the rest of the data. Because of this, we generally ignore results from the last bin given that we are not too worried with what is happening at that ell range anyway. Shown below are plots of BB to BB and EE to BB band power window functions for both the messenger method estimator and the S2hat estimator. Band power window functions for the messenger estimator were calculated with an input signal C_ell = 1 on the sky. This means that the actual input power for deriving band power window functions is 1 * B_ell squared, where B_ell squared contains the effect of the beam. We also ran 6 simulations at each ell with this input power and then took the mean of the output across these six simulations to get the final band power window functions. Summing the window functions across ell values gives us the total suppression factor.<br />
<br />
<br />
[[File:Messenger bb2bb bpwf.png|frame|Figure 4: BB to BB band power window functions using messenger method pure-B estimator.|center]]<br />
[[File:Messenger ee2bb bpwf.png|frame|Figure 5: EE to BB band power window functions using messenger method pure-B estimator.|center]]<br />
[[File:S2hat bb2bb bpwf.png|frame|Figure 6: BB to BB band power window functions using S2hat pure-B estimator|center]]<br />
<br />
<br />
Again, it is visually apparent that there is something odd going on in the last bin for the messenger method estimator.<br />
<br />
Below are plots the variance in each bin across the 100 realizations which were filtered. The exact same simulations were used as input to the two methods compared (signal plus noise simulations at 95 GHz using the 04.00 experiment configuration).<br />
<br />
<br />
[[File:Variance comparison.png|frame|Figure : Comparison of messenger method and S2hat estimator when it comes to the variance in each bin.|center]]<br />
<br />
As shown above, the messenger method beats the S2hat estimator in terms of variance per bin. We believe the main reason for this is that the S2hat estimator uses a simple weighting of 1/N to filter the maps. Since the signal plus noise simulations have a sizeable amount of BB lensing signal contained within them, the fact that we are doing a weiner filter (and thus taking into account the signal covariance) means that we would expect to get a large improvement over an estimator which only knows about noise covariance. For delensed simulations, we would expect to see less of an improvement from the messenger method over the S2hat filtering.</div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=PureB_by_Messenger_Method&diff=9242PureB by Messenger Method2019-09-12T15:22:48Z<p>Michaelray: </p>
<hr />
<div>''Michael Ray, 2019-09-xx''<br />
----<br />
<br />
Dr. Bischoff and myself have been working on a new approach to a pure-B estimator for CMB polarization data. The method was originally proposed in 2012 (Elsner & Wandelt 2012) and has been dubbed the "messenger method" approach to weiner filtering. To make a pure-B weiner filter, we represent the noise covariance matrix as the noise covariance plus the E component of the signal covariance (Bunn & Wandelt 2017), so that the filter treats any E modes as noise. This means the weiner filter equation becomes:<br />
<br />
data<sub>wf</sub> = (S<sub>b</sub>/(S<sub>b</sub>+S<sub>e</sub>+N))*data<br />
<br />
The difficulty in weiner filtering for CMB analysis lies in the fact that we can only easily write down S in the sphereical harmonic basis, and N in the pixel basis. In fact, S is diagonal in spherical harmonic space, and N is diagonal in pixel space. Because of this, we must use a work around in order to weiner filter maps. The general approach of the messenger method is to introduce a new field, t, which is called the "messenger field". This field has covariance T, which is proportional to the identity matrix and represents the homogeneous portion of the noise covariance. Being proportional to the identity matrix, T can be written diagonally in any basis we choose (Elsner and Wandelt 2012).<br />
<br />
Since we have multiple matrices that can only be written in one basis, this suggests some sort of iterative algorithm to solve for a weiner filtered map where we use the messenger field t and its covariance matrix to mediate between pixel and sphereical harmonic space. Elsner and Wandelt lay out this procedure in their paper and they come up with two equations to be solved iteratively for a weiner filtered map. In their equations, there is a lambda parameter which is a scale factor used to artificially increase the covariance T. This is useful in speeding up convergence of the algorithm. At the end of the algorithm, lambda must be reduced to 1 in order to solve for the true weiner filtered map.<br />
<br />
In our implementation, lambda begins at a value of 1300, then on the second iteration it is decreased to 100, and from then on out, lambda -> lambda*eta on each iteration, where eta = 0.825. When lambda reaches 1, our algorithm iterates another 5 times. We have experimented fairly extensively with other cooling schedules for lambda and found that our method gives the best convergence in the least amount of time. We are able to filter a single set of Q and U maps in 34 minutes on a single processor. Shown below is an example of a filtered and unfiltered map for visualization of what our algorithm is doing. The input maps below came from CMBS4 combined simulations at 95 GHz using experiment configuration 04.00. So far, the only maps we have done any work with are at 95 GHz and in this experiment configuration. The algorithm is set up to work with any set of Q and U maps, however, and thus can easily be used at any frequency with any experimental configuration.<br />
<br />
<br />
[[File:Q input map.png|frame|Figure 1: Input Q signal plus noise map. This is zoomed in on the unmasked region of the map.|center]]<br />
[[File:Q output map.png|frame|Figure 2: Output Q signal plus noise map. This has been filtered through our pure B estimator. This map is also zoomed in on the unmasked region although the algorithm extends the map into the unmasked region.|center]]<br />
<br />
<br />
The algorithm takes in a set of two maps, corresponding to a Q and U measurement, and outputs a B only weiner filtered set of Q and U maps. One can visually see that although the input map has a defined edge, the algorithm extends the output map into the unobserved region. This is because the code is coming up with a full sky weiner filtered map and thus any power detected at low ell will result in map fluctuations at large angular scales. It is also visually apparent that the input Q map contains mostly E modes (fluctuations that appear straight up and down or straight across), while the output Q map looks like it's all B modes (diagonal fluctuations). This is exactly what we would hope to see from a pure-B estimator. Note that there is a scale difference between the two plots. This is because the input map contains E signal, so the Q map will have a larger amplitude in each pixel for the input map than the output map which contains only B modes.<br />
<br />
There is still some noise bias in the output map, however we can remove that through using monte carlo methods. There is also a suppression factor involved in the output data which is the reason for the scale difference in plots. This factor is corrected for through the use of band power window functions. Both the noise debiasing and suppression factor correction are made at the power spectrum level only. We are not capable of applying noise and suppression factor corrections to maps.<br />
<br />
Shown below is a plot of 100 spectra which are cleaned signal plus noise maps at 95 GHz taken from the 04.00 CMBS4 experiment configuration. The bins used in this analysis began at ell of 20, and went up through ell of 370 with a bin size of 35. So, bin number zero is ell of 20 - 55. Bin 9 (last bin) is ell of 335 - 370. Also included is the mean of these 100 spectra and the expectation value which was calculated using band power window functions and theory BB spectra.<br />
<br />
<br />
[[File:Filtered specs means vs expectation.png|frame|Figure 3: Band powers of 100 filtered signal plus noise simulations. These were filtered using the messenger method estimator. Also shown is the mean and the expectation value calculated using band power window functions and BB theory spectra.|center]]<br />
<br />
<br />
As one can see, the expectation value for our estimator runs directly through the middle of our simulations and is very close to our mean. It is worth noting that there is definitely something out of the ordinary happening in our last bin. In this bin, we routinely get results that do not agree with the rest of the data. Because of this, we generally ignore results from the last bin given that we are not too worried with what is happening at that ell range anyway. Shown below are plots of BB to BB and EE to BB band power window functions for both the messenger method estimator and the S2hat estimator. Band power window functions for the messenger estimator were calculated with an input signal C_ell = 1 on the sky. This means that the actual input power for deriving band power window functions is 1 * B_ell squared, where B_ell squared contains the effect of the beam. We also ran 6 simulations at each ell with this input power and then took the mean of the output across these six simulations to get the final band power window functions. Summing the window functions across ell values gives us the total suppression factor.<br />
<br />
<br />
[[File:Messenger bb2bb bpwf.png|frame|Figure 4: BB to BB band power window functions using messenger method pure-B estimator.|center]]<br />
[[File:Messenger ee2bb bpwf.png|frame|Figure 5: EE to BB band power window functions using messenger method pure-B estimator.|center]]<br />
[[File:S2hat bb2bb bpwf.png|frame|Figure 6: BB to BB band power window functions using S2hat pure-B estimator|center]]<br />
<br />
<br />
Again, it is visually apparent that there is something odd going on in the last bin for the messenger method estimator.<br />
<br />
Below are plots the variance in each bin across the 100 realizations which were filtered. The exact same simulations were used as input to the two methods compared (signal plus noise simulations at 95 GHz using the 04.00 experiment configuration).<br />
<br />
<br />
[[File:Variance comparison.png|frame|Figure : Comparison of messenger method and S2hat estimator when it comes to the variance in each bin.|center]]<br />
<br />
As shown above, the messenger method beats the S2hat estimator in terms of variance per bin. We believe the main reason for this is that the S2hat estimator uses a simple weighting of 1/N to filter the maps. Since the signal plus noise simulations have a sizeable amount of BB lensing signal contained within them, the fact that we are doing a weiner filter (and thus taking into account the signal covariance) means that we would expect to get a large improvement over an estimator which only knows about noise covariance. For delensed simulations, we would expect to see less of an improvement from the messenger method over the S2hat filtering.</div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=File:Filtered_specs_mean_vs_expectation.png&diff=9241File:Filtered specs mean vs expectation.png2019-09-12T15:11:23Z<p>Michaelray: </p>
<hr />
<div></div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=File:Messenger_bb2bb_bpwf.png&diff=9240File:Messenger bb2bb bpwf.png2019-09-12T15:11:11Z<p>Michaelray: </p>
<hr />
<div></div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=File:Messenger_ee2bb_bpwf.png&diff=9239File:Messenger ee2bb bpwf.png2019-09-12T15:11:02Z<p>Michaelray: </p>
<hr />
<div></div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=File:Q_input_map.png&diff=9238File:Q input map.png2019-09-12T15:10:53Z<p>Michaelray: </p>
<hr />
<div></div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=File:Q_output_map.png&diff=9237File:Q output map.png2019-09-12T15:10:43Z<p>Michaelray: </p>
<hr />
<div></div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=File:Variance_comparison.png&diff=9236File:Variance comparison.png2019-09-12T15:10:26Z<p>Michaelray: Michaelray uploaded a new version of File:Variance comparison.png</p>
<hr />
<div></div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=File:S2hat_bb2bb_bpwf.png&diff=9235File:S2hat bb2bb bpwf.png2019-09-12T15:09:56Z<p>Michaelray: Michaelray uploaded a new version of File:S2hat bb2bb bpwf.png</p>
<hr />
<div></div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=File:S2hat_bb2bb_bpwf.png&diff=9234File:S2hat bb2bb bpwf.png2019-09-12T15:05:43Z<p>Michaelray: </p>
<hr />
<div></div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=File:Variance_comparison.png&diff=9233File:Variance comparison.png2019-09-12T15:04:35Z<p>Michaelray: Michaelray uploaded a new version of File:Variance comparison.png</p>
<hr />
<div></div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=PureB_by_Messenger_Method&diff=9232PureB by Messenger Method2019-09-12T14:05:08Z<p>Michaelray: </p>
<hr />
<div>''Michael Ray, 2019-09-xx''<br />
----<br />
<br />
Dr. Bischoff and myself have been working on a new approach to a pure-B estimator for CMB polarization data. The method was originally proposed in 2012 (Elsner & Wandelt 2012) and has been dubbed the "messenger method" approach to weiner filtering. To make a pure-B weiner filter, we represent the noise covariance matrix as the noise covariance plus the E component of the signal covariance (Bunn & Wandelt 2017), so that the filter treats any E modes as noise. This means the weiner filter equation becomes:<br />
<br />
data<sub>wf</sub> = S<sub>b</sub>/(S<sub>b</sub>+S<sub>e</sub>+N)*data<br />
<br />
The difficulty in weiner filtering for CMB analysis lies in the fact that we can only easily write down S in the sphereical harmonic basis, and N in the pixel basis. In fact, S is diagonal in spherical harmonic space, and N is diagonal in pixel space. Because of this, we must use a work around in order to weiner filter maps. The general approach of the messenger method is to introduce a new field, t, which is called the "messenger field". This field has covariance T, which is proportional to the identity matrix and represents the homogeneous portion of the noise covariance. Being proportional to the identity matrix, T can be written diagonally in any basis we choose (Elsner and Wandelt 2012).<br />
<br />
Since we have multiple matrices that can only be written in one basis, this suggests some sort of iterative algorithm to solve for a weiner filtered map where we use the messenger field t and its covariance matrix to mediate between pixel and sphereical harmonic space. Elsner and Wandelt lay out this procedure in their paper and they come up with two equations to be solved iteratively for a weiner filtered map. In their equations, there is a lambda parameter which is a scale factor used to artificially increase the covariance T. This is useful in speeding up convergence of the algorithm. At the end of the algorithm, lambda must be reduced to 1 in order to solve for the true weiner filtered map.<br />
<br />
In our implementation, lambda begins at a value of 1300, then on the second iteration it is decreased to 100, and from then on out, lambda -> lambda*eta on each iteration, where eta = 0.825. When lambda reaches 1, our algorithm iterates another 5 times. We have experimented fairly extensively with other cooling schedules for lambda and found that our method gives the best convergence in the least amount of time. We are able to filter a single set of Q and U maps in 34 minutes on a single processor. Shown below is an example of a filtered and unfiltered map for visualization of what our algorithm is doing. The input maps below came from CMBS4 combined simulations at 95 GHz using experiment configuration 04.00. So far, the only maps we have done any work with are at 95 GHz and in this experiment configuration. The algorithm is set up to work with any set of Q and U maps, however, and thus can easily be used at any frequency with any experimental configuration.<br />
<br />
<br />
***Put in images of input and output Q maps.***<br />
<br />
<br />
The algorithm takes in a set of two maps, corresponding to a Q and U measurement, and outputs a B only weiner filtered set of Q and U maps. One can visually see that although the input map has a defined edge, the algorithm extends the output map into the unobserved region. This is because the code is coming up with a full sky weiner filtered map and thus any power detected at low ell will result in map fluctuations at large angular scales. It is also visually apparent that the input Q map contains mostly E modes (fluctuations that appear straight up and down or straight across), while the output Q map looks like it's all B modes (diagonal fluctuations). This is exactly what we would hope to see from a pure-B estimator. Note that there is a scale difference between the two plots. This is because the input map contains E signal, so the Q map will have a larger amplitude in each pixel for the input map than the output map which contains only B modes.<br />
<br />
There is still some noise bias in the output map, however we can remove that through using monte carlo methods. There is also a suppression factor involved in the output data which is the reason for the scale difference in plots. This factor is corrected for through the use of band power window functions. Both the noise debiasing and suppression factor correction are made at the power spectrum level only. We are not capable of applying noise and suppression factor corrections to maps.<br />
<br />
Shown below is a plot of 100 spectra which are cleaned signal plus noise maps at 95 GHz taken from the 04.00 CMBS4 experiment configuration. The bins used in this analysis began at ell of 20, and went up through ell of 370 with a bin size of 35. So, bin number zero is ell of 20 - 55. Bin 9 (last bin) is ell of 335 - 370. Also included is the mean of these 100 spectra and the expectation value which was calculated using band power window functions and theory BB spectra.<br />
<br />
<br />
***Put in plots of filtered signal plus noise band powers***<br />
<br />
<br />
As one can see, the expectation value for our estimator runs directly through the middle of our simulations and is very close to our mean. It is worth noting that there is definitely something out of the ordinary happening in our last bin. In this bin, we routinely get results that do not agree with the rest of the data. Because of this, we generally ignore results from the last bin given that we are not too worried with what is happening at that ell range anyway. Shown below are plots of BB to BB and EE to BB band power window functions for both the messenger method estimator and the S2hat estimator. Band power window functions for the messenger estimator were calculated with an input signal C_ell = 1 on the sky. This means that the actual input power for deriving band power window functions is 1 * B_ell squared, where B_ell squared contains the effect of the beam. We also ran 6 simulations at each ell with this input power and then took the mean of the output across these six simulations to get the final band power window functions. Summing the window functions across ell values gives us the total suppression factor.<br />
<br />
<br />
***Put in plots of s2hat and messenger band power window functions***<br />
<br />
<br />
Again, it is visually apparent that there is something odd going on in the last bin for the messenger method estimator.<br />
<br />
Below are plots the variance in each bin across the 100 realizations which were filtered. The exact same simulations were used as input to the two methods compared (signal plus noise simulations at 95 GHz using the 04.00 experiment configuration).<br />
<br />
<br />
[[File:Variance comparison.png|frame|Figure 1: Marginalized likelihood on r for four experiment configurations|center]]<br />
<br />
As shown above, the messenger method beats the S2hat estimator in terms of variance per bin. We believe the main reason for this is that the S2hat estimator uses a simple weighting of 1/N to filter the maps. Since the signal plus noise simulations have a sizeable amount of BB lensing signal contained within them, the fact that we are doing a weiner filter (and thus taking into account the signal covariance) means that we would expect to get a large improvement over an estimator which only knows about noise covariance. For delensed simulations, we would expect to see less of an improvement from the messenger method over the S2hat filtering.</div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=PureB_by_Messenger_Method&diff=9231PureB by Messenger Method2019-09-12T14:04:57Z<p>Michaelray: </p>
<hr />
<div>''Michael Ray, 2019-09-xx''<br />
----<br />
<br />
Dr. Bischoff and myself have been working on a new approach to a pure-B estimator for CMB polarization data. The method was originally proposed in 2012 (Elsner & Wandelt 2012) and has been dubbed the "messenger method" approach to weiner filtering. To make a pure-B weiner filter, we represent the noise covariance matrix as the noise covariance plus the E component of the signal covariance (Bunn & Wandelt 2017), so that the filter treats any E modes as noise. This means the weiner filter equation becomes:<br />
<br />
***Fix equation below***<br />
data<sub>wf</sub> = S<sub>b</sub>/(S<sub>b</sub>+S<sub>e</sub>+N)*data><br />
<br />
The difficulty in weiner filtering for CMB analysis lies in the fact that we can only easily write down S in the sphereical harmonic basis, and N in the pixel basis. In fact, S is diagonal in spherical harmonic space, and N is diagonal in pixel space. Because of this, we must use a work around in order to weiner filter maps. The general approach of the messenger method is to introduce a new field, t, which is called the "messenger field". This field has covariance T, which is proportional to the identity matrix and represents the homogeneous portion of the noise covariance. Being proportional to the identity matrix, T can be written diagonally in any basis we choose (Elsner and Wandelt 2012).<br />
<br />
Since we have multiple matrices that can only be written in one basis, this suggests some sort of iterative algorithm to solve for a weiner filtered map where we use the messenger field t and its covariance matrix to mediate between pixel and sphereical harmonic space. Elsner and Wandelt lay out this procedure in their paper and they come up with two equations to be solved iteratively for a weiner filtered map. In their equations, there is a lambda parameter which is a scale factor used to artificially increase the covariance T. This is useful in speeding up convergence of the algorithm. At the end of the algorithm, lambda must be reduced to 1 in order to solve for the true weiner filtered map.<br />
<br />
In our implementation, lambda begins at a value of 1300, then on the second iteration it is decreased to 100, and from then on out, lambda -> lambda*eta on each iteration, where eta = 0.825. When lambda reaches 1, our algorithm iterates another 5 times. We have experimented fairly extensively with other cooling schedules for lambda and found that our method gives the best convergence in the least amount of time. We are able to filter a single set of Q and U maps in 34 minutes on a single processor. Shown below is an example of a filtered and unfiltered map for visualization of what our algorithm is doing. The input maps below came from CMBS4 combined simulations at 95 GHz using experiment configuration 04.00. So far, the only maps we have done any work with are at 95 GHz and in this experiment configuration. The algorithm is set up to work with any set of Q and U maps, however, and thus can easily be used at any frequency with any experimental configuration.<br />
<br />
<br />
***Put in images of input and output Q maps.***<br />
<br />
<br />
The algorithm takes in a set of two maps, corresponding to a Q and U measurement, and outputs a B only weiner filtered set of Q and U maps. One can visually see that although the input map has a defined edge, the algorithm extends the output map into the unobserved region. This is because the code is coming up with a full sky weiner filtered map and thus any power detected at low ell will result in map fluctuations at large angular scales. It is also visually apparent that the input Q map contains mostly E modes (fluctuations that appear straight up and down or straight across), while the output Q map looks like it's all B modes (diagonal fluctuations). This is exactly what we would hope to see from a pure-B estimator. Note that there is a scale difference between the two plots. This is because the input map contains E signal, so the Q map will have a larger amplitude in each pixel for the input map than the output map which contains only B modes.<br />
<br />
There is still some noise bias in the output map, however we can remove that through using monte carlo methods. There is also a suppression factor involved in the output data which is the reason for the scale difference in plots. This factor is corrected for through the use of band power window functions. Both the noise debiasing and suppression factor correction are made at the power spectrum level only. We are not capable of applying noise and suppression factor corrections to maps.<br />
<br />
Shown below is a plot of 100 spectra which are cleaned signal plus noise maps at 95 GHz taken from the 04.00 CMBS4 experiment configuration. The bins used in this analysis began at ell of 20, and went up through ell of 370 with a bin size of 35. So, bin number zero is ell of 20 - 55. Bin 9 (last bin) is ell of 335 - 370. Also included is the mean of these 100 spectra and the expectation value which was calculated using band power window functions and theory BB spectra.<br />
<br />
<br />
***Put in plots of filtered signal plus noise band powers***<br />
<br />
<br />
As one can see, the expectation value for our estimator runs directly through the middle of our simulations and is very close to our mean. It is worth noting that there is definitely something out of the ordinary happening in our last bin. In this bin, we routinely get results that do not agree with the rest of the data. Because of this, we generally ignore results from the last bin given that we are not too worried with what is happening at that ell range anyway. Shown below are plots of BB to BB and EE to BB band power window functions for both the messenger method estimator and the S2hat estimator. Band power window functions for the messenger estimator were calculated with an input signal C_ell = 1 on the sky. This means that the actual input power for deriving band power window functions is 1 * B_ell squared, where B_ell squared contains the effect of the beam. We also ran 6 simulations at each ell with this input power and then took the mean of the output across these six simulations to get the final band power window functions. Summing the window functions across ell values gives us the total suppression factor.<br />
<br />
<br />
***Put in plots of s2hat and messenger band power window functions***<br />
<br />
<br />
Again, it is visually apparent that there is something odd going on in the last bin for the messenger method estimator.<br />
<br />
Below are plots the variance in each bin across the 100 realizations which were filtered. The exact same simulations were used as input to the two methods compared (signal plus noise simulations at 95 GHz using the 04.00 experiment configuration).<br />
<br />
<br />
[[File:Variance comparison.png|frame|Figure 1: Marginalized likelihood on r for four experiment configurations|center]]<br />
<br />
As shown above, the messenger method beats the S2hat estimator in terms of variance per bin. We believe the main reason for this is that the S2hat estimator uses a simple weighting of 1/N to filter the maps. Since the signal plus noise simulations have a sizeable amount of BB lensing signal contained within them, the fact that we are doing a weiner filter (and thus taking into account the signal covariance) means that we would expect to get a large improvement over an estimator which only knows about noise covariance. For delensed simulations, we would expect to see less of an improvement from the messenger method over the S2hat filtering.</div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=PureB_by_Messenger_Method&diff=9230PureB by Messenger Method2019-09-12T14:01:32Z<p>Michaelray: </p>
<hr />
<div>''Michael Ray, 2019-09-xx''<br />
----<br />
<br />
Dr. Bischoff and myself have been working on a new approach to a pure-B estimator for CMB polarization data. The method was originally proposed in 2012 (Elsner & Wandelt 2012) and has been dubbed the "messenger method" approach to weiner filtering. To make a pure-B weiner filter, we represent the noise covariance matrix as the noise covariance plus the E component of the signal covariance (Bunn & Wandelt 2017), so that the filter treats any E modes as noise. This means the weiner filter equation becomes:<br />
<br />
***Fix equation below***<br />
<data_wf = S_b/(S_b+S_e+N)*data><br />
<br />
The difficulty in weiner filtering for CMB analysis lies in the fact that we can only easily write down S in the sphereical harmonic basis, and N in the pixel basis. In fact, S is diagonal in spherical harmonic space, and N is diagonal in pixel space. Because of this, we must use a work around in order to weiner filter maps. The general approach of the messenger method is to introduce a new field, t, which is called the "messenger field". This field has covariance T, which is proportional to the identity matrix and represents the homogeneous portion of the noise covariance. Being proportional to the identity matrix, T can be written diagonally in any basis we choose (Elsner and Wandelt 2012).<br />
<br />
Since we have multiple matrices that can only be written in one basis, this suggests some sort of iterative algorithm to solve for a weiner filtered map where we use the messenger field t and its covariance matrix to mediate between pixel and sphereical harmonic space. Elsner and Wandelt lay out this procedure in their paper and they come up with two equations to be solved iteratively for a weiner filtered map. In their equations, there is a lambda parameter which is a scale factor used to artificially increase the covariance T. This is useful in speeding up convergence of the algorithm. At the end of the algorithm, lambda must be reduced to 1 in order to solve for the true weiner filtered map.<br />
<br />
In our implementation, lambda begins at a value of 1300, then on the second iteration it is decreased to 100, and from then on out, lambda -> lambda*eta on each iteration, where eta = 0.825. When lambda reaches 1, our algorithm iterates another 5 times. We have experimented fairly extensively with other cooling schedules for lambda and found that our method gives the best convergence in the least amount of time. We are able to filter a single set of Q and U maps in 34 minutes on a single processor. Shown below is an example of a filtered and unfiltered map for visualization of what our algorithm is doing. The input maps below came from CMBS4 combined simulations at 95 GHz using experiment configuration 04.00. So far, the only maps we have done any work with are at 95 GHz and in this experiment configuration. The algorithm is set up to work with any set of Q and U maps, however, and thus can easily be used at any frequency with any experimental configuration.<br />
<br />
<br />
***Put in images of input and output Q maps.***<br />
<br />
<br />
The algorithm takes in a set of two maps, corresponding to a Q and U measurement, and outputs a B only weiner filtered set of Q and U maps. One can visually see that although the input map has a defined edge, the algorithm extends the output map into the unobserved region. This is because the code is coming up with a full sky weiner filtered map and thus any power detected at low ell will result in map fluctuations at large angular scales. It is also visually apparent that the input Q map contains mostly E modes (fluctuations that appear straight up and down or straight across), while the output Q map looks like it's all B modes (diagonal fluctuations). This is exactly what we would hope to see from a pure-B estimator. Note that there is a scale difference between the two plots. This is because the input map contains E signal, so the Q map will have a larger amplitude in each pixel for the input map than the output map which contains only B modes.<br />
<br />
There is still some noise bias in the output map, however we can remove that through using monte carlo methods. There is also a suppression factor involved in the output data which is the reason for the scale difference in plots. This factor is corrected for through the use of band power window functions. Both the noise debiasing and suppression factor correction are made at the power spectrum level only. We are not capable of applying noise and suppression factor corrections to maps.<br />
<br />
Shown below is a plot of 100 spectra which are cleaned signal plus noise maps at 95 GHz taken from the 04.00 CMBS4 experiment configuration. The bins used in this analysis began at ell of 20, and went up through ell of 370 with a bin size of 35. So, bin number zero is ell of 20 - 55. Bin 9 (last bin) is ell of 335 - 370. Also included is the mean of these 100 spectra and the expectation value which was calculated using band power window functions and theory BB spectra.<br />
<br />
<br />
***Put in plots of filtered signal plus noise band powers***<br />
<br />
<br />
As one can see, the expectation value for our estimator runs directly through the middle of our simulations and is very close to our mean. It is worth noting that there is definitely something out of the ordinary happening in our last bin. In this bin, we routinely get results that do not agree with the rest of the data. Because of this, we generally ignore results from the last bin given that we are not too worried with what is happening at that ell range anyway. Shown below are plots of BB to BB and EE to BB band power window functions for both the messenger method estimator and the S2hat estimator. Band power window functions for the messenger estimator were calculated with an input signal C_ell = 1 on the sky. This means that the actual input power for deriving band power window functions is 1 * B_ell squared, where B_ell squared contains the effect of the beam. We also ran 6 simulations at each ell with this input power and then took the mean of the output across these six simulations to get the final band power window functions. Summing the window functions across ell values gives us the total suppression factor.<br />
<br />
<br />
***Put in plots of s2hat and messenger band power window functions***<br />
<br />
<br />
Again, it is visually apparent that there is something odd going on in the last bin for the messenger method estimator.<br />
<br />
Below are plots the variance in each bin across the 100 realizations which were filtered. The exact same simulations were used as input to the two methods compared (signal plus noise simulations at 95 GHz using the 04.00 experiment configuration).<br />
<br />
<br />
[[File:Variance comparison.png|frame|Figure 1: Marginalized likelihood on r for four experiment configurations|center]]<br />
<br />
As shown above, the messenger method beats the S2hat estimator in terms of variance per bin. We believe the main reason for this is that the S2hat estimator uses a simple weighting of 1/N to filter the maps. Since the signal plus noise simulations have a sizeable amount of BB lensing signal contained within them, the fact that we are doing a weiner filter (and thus taking into account the signal covariance) means that we would expect to get a large improvement over an estimator which only knows about noise covariance. For delensed simulations, we would expect to see less of an improvement from the messenger method over the S2hat filtering.</div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=PureB_by_Messenger_Method&diff=9229PureB by Messenger Method2019-09-12T14:01:08Z<p>Michaelray: </p>
<hr />
<div>''Michael Ray, 2019-09-xx''<br />
----<br />
<br />
Dr. Bischoff and myself have been working on a new approach to a pure-B estimator for CMB polarization data. The method was originally proposed in 2012 (Elsner & Wandelt 2012) and has been dubbed the "messenger method" approach to weiner filtering. To make a pure-B weiner filter, we represent the noise covariance matrix as the noise covariance plus the E component of the signal covariance (Bunn & Wandelt 2017), so that the filter treats any E modes as noise. This means the weiner filter equation becomes:<br />
<br />
***Fix equation below***<br />
<data_wf = S_b/(S_b+S_e+N)*data><br />
<br />
The difficulty in weiner filtering for CMB analysis lies in the fact that we can only easily write down S in the sphereical harmonic basis, and N in the pixel basis. In fact, S is diagonal in spherical harmonic space, and N is diagonal in pixel space. Because of this, we must use a work around in order to weiner filter maps. The general approach of the messenger method is to introduce a new field, t, which is called the "messenger field". This field has covariance T, which is proportional to the identity matrix and represents the homogeneous portion of the noise covariance. Being proportional to the identity matrix, T can be written diagonally in any basis we choose (Elsner and Wandelt 2012).<br />
<br />
Since we have multiple matrices that can only be written in one basis, this suggests some sort of iterative algorithm to solve for a weiner filtered map where we use the messenger field t and its covariance matrix to mediate between pixel and sphereical harmonic space. Elsner and Wandelt lay out this procedure in their paper and they come up with two equations to be solved iteratively for a weiner filtered map. In their equations, there is a lambda parameter which is a scale factor used to artificially increase the covariance T. This is useful in speeding up convergence of the algorithm. At the end of the algorithm, lambda must be reduced to 1 in order to solve for the true weiner filtered map.<br />
<br />
In our implementation, lambda begins at a value of 1300, then on the second iteration it is decreased to 100, and from then on out, lambda -> lambda*eta on each iteration, where eta = 0.825. When lambda reaches 1, our algorithm iterates another 5 times. We have experimented fairly extensively with other cooling schedules for lambda and found that our method gives the best convergence in the least amount of time. We are able to filter a single set of Q and U maps in 34 minutes on a single processor. Shown below is an example of a filtered and unfiltered map for visualization of what our algorithm is doing. The input maps below came from CMBS4 combined simulations at 95 GHz using experiment configuration 04.00. So far, the only maps we have done any work with are at 95 GHz and in this experiment configuration. The algorithm is set up to work with any set of Q and U maps, however, and thus can easily be used at any frequency with any experimental configuration.<br />
<br />
<br />
***Put in images of input and output Q maps.***<br />
<br />
<br />
The algorithm takes in a set of two maps, corresponding to a Q and U measurement, and outputs a B only weiner filtered set of Q and U maps. One can visually see that although the input map has a defined edge, the algorithm extends the output map into the unobserved region. This is because the code is coming up with a full sky weiner filtered map and thus any power detected at low ell will result in map fluctuations at large angular scales. It is also visually apparent that the input Q map contains mostly E modes (fluctuations that appear straight up and down or straight across), while the output Q map looks like it's all B modes (diagonal fluctuations). This is exactly what we would hope to see from a pure-B estimator. Note that there is a scale difference between the two plots. This is because the input map contains E signal, so the Q map will have a larger amplitude in each pixel for the input map than the output map which contains only B modes.<br />
<br />
There is still some noise bias in the output map, however we can remove that through using monte carlo methods. There is also a suppression factor involved in the output data which is the reason for the scale difference in plots. This factor is corrected for through the use of band power window functions. Both the noise debiasing and suppression factor correction are made at the power spectrum level only. We are not capable of applying noise and suppression factor corrections to maps.<br />
<br />
Shown below is a plot of 100 spectra which are cleaned signal plus noise maps at 95 GHz taken from the 04.00 CMBS4 experiment configuration. The bins used in this analysis began at ell of 20, and went up through ell of 370 with a bin size of 35. So, bin number zero is ell of 20 - 55. Bin 9 (last bin) is ell of 335 - 370. Also included is the mean of these 100 spectra and the expectation value which was calculated using band power window functions and theory BB spectra.<br />
<br />
<br />
***Put in plots of filtered signal plus noise band powers***<br />
<br />
<br />
As one can see, the expectation value for our estimator runs directly through the middle of our simulations and is very close to our mean. It is worth noting that there is definitely something out of the ordinary happening in our last bin. In this bin, we routinely get results that do not agree with the rest of the data. Because of this, we generally ignore results from the last bin given that we are not too worried with what is happening at that ell range anyway. Shown below are plots of BB to BB and EE to BB band power window functions for both the messenger method estimator and the S2hat estimator. Band power window functions for the messenger estimator were calculated with an input signal C_ell = 1 on the sky. This means that the actual input power for deriving band power window functions is 1 * B_ell squared, where B_ell squared contains the effect of the beam. We also ran 6 simulations at each ell with this input power and then took the mean of the output across these six simulations to get the final band power window functions. Summing the window functions across ell values gives us the total suppression factor.<br />
<br />
<br />
***Put in plots of s2hat and messenger band power window functions***<br />
<br />
<br />
Again, it is visually apparent that there is something odd going on in the last bin for the messenger method estimator.<br />
<br />
Below are plots the variance in each bin across the 100 realizations which were filtered. The exact same simulations were used as input to the two methods compared (signal plus noise simulations at 95 GHz using the 04.00 experiment configuration).<br />
<br />
<br />
[[File:Variance Comparison.png|frame|Figure 1: Marginalized likelihood on r for four experiment configurations|center]]<br />
<br />
As shown above, the messenger method beats the S2hat estimator in terms of variance per bin. We believe the main reason for this is that the S2hat estimator uses a simple weighting of 1/N to filter the maps. Since the signal plus noise simulations have a sizeable amount of BB lensing signal contained within them, the fact that we are doing a weiner filter (and thus taking into account the signal covariance) means that we would expect to get a large improvement over an estimator which only knows about noise covariance. For delensed simulations, we would expect to see less of an improvement from the messenger method over the S2hat filtering.</div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=File:Variance_comparison.png&diff=9228File:Variance comparison.png2019-09-12T02:45:36Z<p>Michaelray: </p>
<hr />
<div></div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=PureB_by_Messenger_Method&diff=9227PureB by Messenger Method2019-09-12T02:43:16Z<p>Michaelray: </p>
<hr />
<div>''Michael Ray, 2019-09-xx''<br />
----<br />
<br />
Dr. Bischoff and myself have been working on a new approach to a pure-B estimator for CMB polarization data. The method was originally proposed in 2012 (Elsner & Wandelt 2012) and has been dubbed the "messenger method" approach to weiner filtering. To make a pure-B weiner filter, we represent the noise covariance matrix as the noise covariance plus the E component of the signal covariance (Bunn & Wandelt 2017), so that the filter treats any E modes as noise. This means the weiner filter equation becomes:<br />
<br />
***Fix equation below***<br />
<data_wf = S_b/(S_b+S_e+N)*data><br />
<br />
The difficulty in weiner filtering for CMB analysis lies in the fact that we can only easily write down S in the sphereical harmonic basis, and N in the pixel basis. In fact, S is diagonal in spherical harmonic space, and N is diagonal in pixel space. Because of this, we must use a work around in order to weiner filter maps. The general approach of the messenger method is to introduce a new field, t, which is called the "messenger field". This field has covariance T, which is proportional to the identity matrix and represents the homogeneous portion of the noise covariance. Being proportional to the identity matrix, T can be written diagonally in any basis we choose (Elsner and Wandelt 2012).<br />
<br />
Since we have multiple matrices that can only be written in one basis, this suggests some sort of iterative algorithm to solve for a weiner filtered map where we use the messenger field t and its covariance matrix to mediate between pixel and sphereical harmonic space. Elsner and Wandelt lay out this procedure in their paper and they come up with two equations to be solved iteratively for a weiner filtered map. In their equations, there is a lambda parameter which is a scale factor used to artificially increase the covariance T. This is useful in speeding up convergence of the algorithm. At the end of the algorithm, lambda must be reduced to 1 in order to solve for the true weiner filtered map.<br />
<br />
In our implementation, lambda begins at a value of 1300, then on the second iteration it is decreased to 100, and from then on out, lambda -> lambda*eta on each iteration, where eta = 0.825. When lambda reaches 1, our algorithm iterates another 5 times. We have experimented fairly extensively with other cooling schedules for lambda and found that our method gives the best convergence in the least amount of time. We are able to filter a single set of Q and U maps in 34 minutes on a single processor. Shown below is an example of a filtered and unfiltered map for visualization of what our algorithm is doing. The input maps below came from CMBS4 combined simulations at 95 GHz using experiment configuration 04.00. So far, the only maps we have done any work with are at 95 GHz and in this experiment configuration. The algorithm is set up to work with any set of Q and U maps, however, and thus can easily be used at any frequency with any experimental configuration.<br />
<br />
<br />
***Put in images of input and output Q maps.***<br />
<br />
<br />
The algorithm takes in a set of two maps, corresponding to a Q and U measurement, and outputs a B only weiner filtered set of Q and U maps. One can visually see that although the input map has a defined edge, the algorithm extends the output map into the unobserved region. This is because the code is coming up with a full sky weiner filtered map and thus any power detected at low ell will result in map fluctuations at large angular scales. It is also visually apparent that the input Q map contains mostly E modes (fluctuations that appear straight up and down or straight across), while the output Q map looks like it's all B modes (diagonal fluctuations). This is exactly what we would hope to see from a pure-B estimator. Note that there is a scale difference between the two plots. This is because the input map contains E signal, so the Q map will have a larger amplitude in each pixel for the input map than the output map which contains only B modes.<br />
<br />
There is still some noise bias in the output map, however we can remove that through using monte carlo methods. There is also a suppression factor involved in the output data which is the reason for the scale difference in plots. This factor is corrected for through the use of band power window functions. Both the noise debiasing and suppression factor correction are made at the power spectrum level only. We are not capable of applying noise and suppression factor corrections to maps.<br />
<br />
Shown below is a plot of 100 spectra which are cleaned signal plus noise maps at 95 GHz taken from the 04.00 CMBS4 experiment configuration. The bins used in this analysis began at ell of 20, and went up through ell of 370 with a bin size of 35. So, bin number zero is ell of 20 - 55. Bin 9 (last bin) is ell of 335 - 370. Also included is the mean of these 100 spectra and the expectation value which was calculated using band power window functions and theory BB spectra.<br />
<br />
<br />
***Put in plots of filtered signal plus noise band powers***<br />
<br />
<br />
As one can see, the expectation value for our estimator runs directly through the middle of our simulations and is very close to our mean. It is worth noting that there is definitely something out of the ordinary happening in our last bin. In this bin, we routinely get results that do not agree with the rest of the data. Because of this, we generally ignore results from the last bin given that we are not too worried with what is happening at that ell range anyway. Shown below are plots of BB to BB and EE to BB band power window functions for both the messenger method estimator and the S2hat estimator. Band power window functions for the messenger estimator were calculated with an input signal C_ell = 1 on the sky. This means that the actual input power for deriving band power window functions is 1 * B_ell squared, where B_ell squared contains the effect of the beam. We also ran 6 simulations at each ell with this input power and then took the mean of the output across these six simulations to get the final band power window functions. Summing the window functions across ell values gives us the total suppression factor.<br />
<br />
<br />
***Put in plots of s2hat and messenger band power window functions***<br />
<br />
<br />
Again, it is visually apparent that there is something odd going on in the last bin for the messenger method estimator.<br />
<br />
Below are plots the variance in each bin across the 100 realizations which were filtered. The exact same simulations were used as input to the two methods compared (signal plus noise simulations at 95 GHz using the 04.00 experiment configuration).<br />
<br />
<br />
[[File:/Users/Michaelray/Documents/Bischoff\ Research/Messenger\ Method/Comparison\ to\ Other\ Estimators/Posting_figs/variance_comparison.png|frame|Figure 1: Marginalized likelihood on r for four experiment configurations|center]]<br />
<br />
As shown above, the messenger method beats the S2hat estimator in terms of variance per bin. We believe the main reason for this is that the S2hat estimator uses a simple weighting of 1/N to filter the maps. Since the signal plus noise simulations have a sizeable amount of BB lensing signal contained within them, the fact that we are doing a weiner filter (and thus taking into account the signal covariance) means that we would expect to get a large improvement over an estimator which only knows about noise covariance. For delensed simulations, we would expect to see less of an improvement from the messenger method over the S2hat filtering.</div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=PureB_by_Messenger_Method&diff=9226PureB by Messenger Method2019-09-11T01:00:15Z<p>Michaelray: </p>
<hr />
<div>''Michael Ray, 2019-09-xx''<br />
----<br />
<br />
Dr. Bischoff and myself have been working on a new approach to a pure-B estimator for CMB polarization data. The method was originally proposed in 2012 (Elsner & Wandelt 2012) and has been dubbed the "messenger method" approach to weiner filtering. To make a pure-B weiner filter, we represent the noise covariance matrix as the noise covariance plus the E component of the signal covariance (Bunn & Wandelt 2017), so that the filter treats any E modes as noise. This means the weiner filter equation becomes:<br />
<br />
***Fix equation below***<br />
<data_wf = S_b/(S_b+S_e+N)*data><br />
<br />
The difficulty in weiner filtering for CMB analysis lies in the fact that we can only easily write down S in the sphereical harmonic basis, and N in the pixel basis. In fact, S is diagonal in spherical harmonic space, and N is diagonal in pixel space. Because of this, we must use a work around in order to weiner filter maps. The general approach of the messenger method is to introduce a new field, t, which is called the "messenger field". This field has covariance T, which is proportional to the identity matrix and represents the homogeneous portion of the noise covariance. Being proportional to the identity matrix, T can be written diagonally in any basis we choose (Elsner and Wandelt 2012).<br />
<br />
Since we have multiple matrices that can only be written in one basis, this suggests some sort of iterative algorithm to solve for a weiner filtered map where we use the messenger field t and its covariance matrix to mediate between pixel and sphereical harmonic space. Elsner and Wandelt lay out this procedure in their paper and they come up with two equations to be solved iteratively for a weiner filtered map. In their equations, there is a lambda parameter which is a scale factor used to artificially increase the covariance T. This is useful in speeding up convergence of the algorithm. At the end of the algorithm, lambda must be reduced to 1 in order to solve for the true weiner filtered map.<br />
<br />
In our implementation, lambda begins at a value of 1300, then on the second iteration it is decreased to 100, and from then on out, lambda -> lambda*eta on each iteration, where eta = 0.825. When lambda reaches 1, our algorithm iterates another 5 times. We have experimented fairly extensively with other cooling schedules for lambda and found that our method gives the best convergence in the least amount of time. We are able to filter a single set of Q and U maps in 34 minutes on a single processor. Shown below is an example of a filtered and unfiltered map for visualization of what our algorithm is doing. The input maps below came from CMBS4 combined simulations at 95 GHz using experiment configuration 04.00. So far, the only maps we have done any work with are at 95 GHz and in this experiment configuration. The algorithm is set up to work with any set of Q and U maps, however, and thus can easily be used at any frequency with any experimental configuration.<br />
<br />
<br />
***Put in images of input and output Q maps.***<br />
<br />
<br />
The algorithm takes in a set of two maps, corresponding to a Q and U measurement, and outputs a B only weiner filtered set of Q and U maps. One can visually see that although the input map has a defined edge, the algorithm extends the output map into the unobserved region. This is because the code is coming up with a full sky weiner filtered map and thus any power detected at low ell will result in map fluctuations at large angular scales. It is also visually apparent that the input Q map contains mostly E modes (fluctuations that appear straight up and down or straight across), while the output Q map looks like it's all B modes (diagonal fluctuations). This is exactly what we would hope to see from a pure-B estimator. Note that there is a scale difference between the two plots. This is because the input map contains E signal, so the Q map will have a larger amplitude in each pixel for the input map than the output map which contains only B modes.<br />
<br />
There is still some noise bias in the output map, however we can remove that through using monte carlo methods. There is also a suppression factor involved in the output data which is the reason for the scale difference in plots. This factor is corrected for through the use of band power window functions. Both the noise debiasing and suppression factor correction are made at the power spectrum level only. We are not capable of applying noise and suppression factor corrections to maps.<br />
<br />
Shown below is a plot of 100 spectra which are cleaned signal plus noise maps at 95 GHz taken from the 04.00 CMBS4 experiment configuration. The bins used in this analysis began at ell of 20, and went up through ell of 370 with a bin size of 35. So, bin number zero is ell of 20 - 55. Bin 9 (last bin) is ell of 335 - 370. Also included is the mean of these 100 spectra and the expectation value which was calculated using band power window functions and theory BB spectra.<br />
<br />
<br />
***Put in plots of filtered signal plus noise band powers***<br />
<br />
<br />
As one can see, the expectation value for our estimator runs directly through the middle of our simulations and is very close to our mean. It is worth noting that there is definitely something out of the ordinary happening in our last bin. In this bin, we routinely get results that do not agree with the rest of the data. Because of this, we generally ignore results from the last bin given that we are not too worried with what is happening at that ell range anyway. Shown below are plots of BB to BB and EE to BB band power window functions for both the messenger method estimator and the S2hat estimator. Band power window functions for the messenger estimator were calculated with an input signal C_ell = 1 on the sky. This means that the actual input power for deriving band power window functions is 1 * B_ell squared, where B_ell squared contains the effect of the beam. We also ran 6 simulations at each ell with this input power and then took the mean of the output across these six simulations to get the final band power window functions. Summing the window functions across ell values gives us the total suppression factor.<br />
<br />
<br />
***Put in plots of s2hat and messenger band power window functions***<br />
<br />
<br />
Again, it is visually apparent that there is something odd going on in the last bin for the messenger method estimator.<br />
<br />
Below are plots the variance in each bin across the 100 realizations which were filtered. The exact same simulations were used as input to the two methods compared (signal plus noise simulations at 95 GHz using the 04.00 experiment configuration).<br />
<br />
<br />
***Put in plots of variances***<br />
<br />
<br />
As shown above, the messenger method beats the S2hat estimator in terms of variance per bin. We believe the main reason for this is that the S2hat estimator uses a simple weighting of 1/N to filter the maps. Since the signal plus noise simulations have a sizeable amount of BB lensing signal contained within them, the fact that we are doing a weiner filter (and thus taking into account the signal covariance) means that we would expect to get a large improvement over an estimator which only knows about noise covariance. For delensed simulations, we would expect to see less of an improvement from the messenger method over the S2hat filtering.</div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=PureB_by_Messenger_Method&diff=9225PureB by Messenger Method2019-09-11T00:59:55Z<p>Michaelray: </p>
<hr />
<div>''Michael Ray, 2019-09-xx''<br />
----<br />
<br />
Dr. Bischoff and myself have been working on a new approach to a pure-B estimator for CMB polarization data. The method was originally proposed in 2012 (Elsner & Wandelt 2012) and has been dubbed the "messenger method" approach to weiner filtering. To make a pure-B weiner filter, we represent the noise covariance matrix as the noise covariance plus the E component of the signal covariance (Bunn & Wandelt 2017), so that the filter treats any E modes as noise. This means the weiner filter equation becomes:<br />
<br />
<eqn data_wf = S_b/(S_b+S_e+N)*data><br />
<br />
The difficulty in weiner filtering for CMB analysis lies in the fact that we can only easily write down S in the sphereical harmonic basis, and N in the pixel basis. In fact, S is diagonal in spherical harmonic space, and N is diagonal in pixel space. Because of this, we must use a work around in order to weiner filter maps. The general approach of the messenger method is to introduce a new field, t, which is called the "messenger field". This field has covariance T, which is proportional to the identity matrix and represents the homogeneous portion of the noise covariance. Being proportional to the identity matrix, T can be written diagonally in any basis we choose (Elsner and Wandelt 2012).<br />
<br />
Since we have multiple matrices that can only be written in one basis, this suggests some sort of iterative algorithm to solve for a weiner filtered map where we use the messenger field t and its covariance matrix to mediate between pixel and sphereical harmonic space. Elsner and Wandelt lay out this procedure in their paper and they come up with two equations to be solved iteratively for a weiner filtered map. In their equations, there is a lambda parameter which is a scale factor used to artificially increase the covariance T. This is useful in speeding up convergence of the algorithm. At the end of the algorithm, lambda must be reduced to 1 in order to solve for the true weiner filtered map.<br />
<br />
In our implementation, lambda begins at a value of 1300, then on the second iteration it is decreased to 100, and from then on out, lambda -> lambda*eta on each iteration, where eta = 0.825. When lambda reaches 1, our algorithm iterates another 5 times. We have experimented fairly extensively with other cooling schedules for lambda and found that our method gives the best convergence in the least amount of time. We are able to filter a single set of Q and U maps in 34 minutes on a single processor. Shown below is an example of a filtered and unfiltered map for visualization of what our algorithm is doing. The input maps below came from CMBS4 combined simulations at 95 GHz using experiment configuration 04.00. So far, the only maps we have done any work with are at 95 GHz and in this experiment configuration. The algorithm is set up to work with any set of Q and U maps, however, and thus can easily be used at any frequency with any experimental configuration.<br />
<br />
<br />
***Put in images of input and output Q maps.***<br />
<br />
<br />
The algorithm takes in a set of two maps, corresponding to a Q and U measurement, and outputs a B only weiner filtered set of Q and U maps. One can visually see that although the input map has a defined edge, the algorithm extends the output map into the unobserved region. This is because the code is coming up with a full sky weiner filtered map and thus any power detected at low ell will result in map fluctuations at large angular scales. It is also visually apparent that the input Q map contains mostly E modes (fluctuations that appear straight up and down or straight across), while the output Q map looks like it's all B modes (diagonal fluctuations). This is exactly what we would hope to see from a pure-B estimator. Note that there is a scale difference between the two plots. This is because the input map contains E signal, so the Q map will have a larger amplitude in each pixel for the input map than the output map which contains only B modes.<br />
<br />
There is still some noise bias in the output map, however we can remove that through using monte carlo methods. There is also a suppression factor involved in the output data which is the reason for the scale difference in plots. This factor is corrected for through the use of band power window functions. Both the noise debiasing and suppression factor correction are made at the power spectrum level only. We are not capable of applying noise and suppression factor corrections to maps.<br />
<br />
Shown below is a plot of 100 spectra which are cleaned signal plus noise maps at 95 GHz taken from the 04.00 CMBS4 experiment configuration. The bins used in this analysis began at ell of 20, and went up through ell of 370 with a bin size of 35. So, bin number zero is ell of 20 - 55. Bin 9 (last bin) is ell of 335 - 370. Also included is the mean of these 100 spectra and the expectation value which was calculated using band power window functions and theory BB spectra.<br />
<br />
<br />
***Put in plots of filtered signal plus noise band powers***<br />
<br />
<br />
As one can see, the expectation value for our estimator runs directly through the middle of our simulations and is very close to our mean. It is worth noting that there is definitely something out of the ordinary happening in our last bin. In this bin, we routinely get results that do not agree with the rest of the data. Because of this, we generally ignore results from the last bin given that we are not too worried with what is happening at that ell range anyway. Shown below are plots of BB to BB and EE to BB band power window functions for both the messenger method estimator and the S2hat estimator. Band power window functions for the messenger estimator were calculated with an input signal C_ell = 1 on the sky. This means that the actual input power for deriving band power window functions is 1 * B_ell squared, where B_ell squared contains the effect of the beam. We also ran 6 simulations at each ell with this input power and then took the mean of the output across these six simulations to get the final band power window functions. Summing the window functions across ell values gives us the total suppression factor.<br />
<br />
<br />
***Put in plots of s2hat and messenger band power window functions***<br />
<br />
<br />
Again, it is visually apparent that there is something odd going on in the last bin for the messenger method estimator.<br />
<br />
Below are plots the variance in each bin across the 100 realizations which were filtered. The exact same simulations were used as input to the two methods compared (signal plus noise simulations at 95 GHz using the 04.00 experiment configuration).<br />
<br />
<br />
***Put in plots of variances***<br />
<br />
<br />
As shown above, the messenger method beats the S2hat estimator in terms of variance per bin. We believe the main reason for this is that the S2hat estimator uses a simple weighting of 1/N to filter the maps. Since the signal plus noise simulations have a sizeable amount of BB lensing signal contained within them, the fact that we are doing a weiner filter (and thus taking into account the signal covariance) means that we would expect to get a large improvement over an estimator which only knows about noise covariance. For delensed simulations, we would expect to see less of an improvement from the messenger method over the S2hat filtering.</div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=PureB_by_Messenger_Method&diff=9224PureB by Messenger Method2019-09-11T00:59:36Z<p>Michaelray: </p>
<hr />
<div>''Michael Ray, 2019-09-xx''<br />
----<br />
<br />
Dr. Bischoff and myself have been working on a new approach to a pure-B estimator for CMB polarization data. The method was originally proposed in 2012 (Elsner & Wandelt 2012) and has been dubbed the "messenger method" approach to weiner filtering. To make a pure-B weiner filter, we represent the noise covariance matrix as the noise covariance plus the E component of the signal covariance (Bunn & Wandelt 2017), so that the filter treats any E modes as noise. This means the weiner filter equation becomes:<br />
<br />
<data_wf = S_b/(S_b+S_e+N)*data><br />
<br />
The difficulty in weiner filtering for CMB analysis lies in the fact that we can only easily write down S in the sphereical harmonic basis, and N in the pixel basis. In fact, S is diagonal in spherical harmonic space, and N is diagonal in pixel space. Because of this, we must use a work around in order to weiner filter maps. The general approach of the messenger method is to introduce a new field, t, which is called the "messenger field". This field has covariance T, which is proportional to the identity matrix and represents the homogeneous portion of the noise covariance. Being proportional to the identity matrix, T can be written diagonally in any basis we choose (Elsner and Wandelt 2012).<br />
<br />
Since we have multiple matrices that can only be written in one basis, this suggests some sort of iterative algorithm to solve for a weiner filtered map where we use the messenger field t and its covariance matrix to mediate between pixel and sphereical harmonic space. Elsner and Wandelt lay out this procedure in their paper and they come up with two equations to be solved iteratively for a weiner filtered map. In their equations, there is a lambda parameter which is a scale factor used to artificially increase the covariance T. This is useful in speeding up convergence of the algorithm. At the end of the algorithm, lambda must be reduced to 1 in order to solve for the true weiner filtered map.<br />
<br />
In our implementation, lambda begins at a value of 1300, then on the second iteration it is decreased to 100, and from then on out, lambda -> lambda*eta on each iteration, where eta = 0.825. When lambda reaches 1, our algorithm iterates another 5 times. We have experimented fairly extensively with other cooling schedules for lambda and found that our method gives the best convergence in the least amount of time. We are able to filter a single set of Q and U maps in 34 minutes on a single processor. Shown below is an example of a filtered and unfiltered map for visualization of what our algorithm is doing. The input maps below came from CMBS4 combined simulations at 95 GHz using experiment configuration 04.00. So far, the only maps we have done any work with are at 95 GHz and in this experiment configuration. The algorithm is set up to work with any set of Q and U maps, however, and thus can easily be used at any frequency with any experimental configuration.<br />
<br />
<br />
***Put in images of input and output Q maps.***<br />
<br />
<br />
The algorithm takes in a set of two maps, corresponding to a Q and U measurement, and outputs a B only weiner filtered set of Q and U maps. One can visually see that although the input map has a defined edge, the algorithm extends the output map into the unobserved region. This is because the code is coming up with a full sky weiner filtered map and thus any power detected at low ell will result in map fluctuations at large angular scales. It is also visually apparent that the input Q map contains mostly E modes (fluctuations that appear straight up and down or straight across), while the output Q map looks like it's all B modes (diagonal fluctuations). This is exactly what we would hope to see from a pure-B estimator. Note that there is a scale difference between the two plots. This is because the input map contains E signal, so the Q map will have a larger amplitude in each pixel for the input map than the output map which contains only B modes.<br />
<br />
There is still some noise bias in the output map, however we can remove that through using monte carlo methods. There is also a suppression factor involved in the output data which is the reason for the scale difference in plots. This factor is corrected for through the use of band power window functions. Both the noise debiasing and suppression factor correction are made at the power spectrum level only. We are not capable of applying noise and suppression factor corrections to maps.<br />
<br />
Shown below is a plot of 100 spectra which are cleaned signal plus noise maps at 95 GHz taken from the 04.00 CMBS4 experiment configuration. The bins used in this analysis began at ell of 20, and went up through ell of 370 with a bin size of 35. So, bin number zero is ell of 20 - 55. Bin 9 (last bin) is ell of 335 - 370. Also included is the mean of these 100 spectra and the expectation value which was calculated using band power window functions and theory BB spectra.<br />
<br />
<br />
***Put in plots of filtered signal plus noise band powers***<br />
<br />
<br />
As one can see, the expectation value for our estimator runs directly through the middle of our simulations and is very close to our mean. It is worth noting that there is definitely something out of the ordinary happening in our last bin. In this bin, we routinely get results that do not agree with the rest of the data. Because of this, we generally ignore results from the last bin given that we are not too worried with what is happening at that ell range anyway. Shown below are plots of BB to BB and EE to BB band power window functions for both the messenger method estimator and the S2hat estimator. Band power window functions for the messenger estimator were calculated with an input signal C_ell = 1 on the sky. This means that the actual input power for deriving band power window functions is 1 * B_ell squared, where B_ell squared contains the effect of the beam. We also ran 6 simulations at each ell with this input power and then took the mean of the output across these six simulations to get the final band power window functions. Summing the window functions across ell values gives us the total suppression factor.<br />
<br />
<br />
***Put in plots of s2hat and messenger band power window functions***<br />
<br />
<br />
Again, it is visually apparent that there is something odd going on in the last bin for the messenger method estimator.<br />
<br />
Below are plots the variance in each bin across the 100 realizations which were filtered. The exact same simulations were used as input to the two methods compared (signal plus noise simulations at 95 GHz using the 04.00 experiment configuration).<br />
<br />
<br />
***Put in plots of variances***<br />
<br />
<br />
As shown above, the messenger method beats the S2hat estimator in terms of variance per bin. We believe the main reason for this is that the S2hat estimator uses a simple weighting of 1/N to filter the maps. Since the signal plus noise simulations have a sizeable amount of BB lensing signal contained within them, the fact that we are doing a weiner filter (and thus taking into account the signal covariance) means that we would expect to get a large improvement over an estimator which only knows about noise covariance. For delensed simulations, we would expect to see less of an improvement from the messenger method over the S2hat filtering.</div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=PureB_by_Messenger_Method&diff=9223PureB by Messenger Method2019-09-11T00:51:04Z<p>Michaelray: </p>
<hr />
<div>''Michael Ray, 2019-09-xx''<br />
----<br />
<br />
Dr. Bischoff and myself have been working on a new approach to a pure-B estimator for CMB polarization data. The method was originally proposed in 2012 (Elsner & Wandelt 2012) and has been dubbed the "messenger method" approach to weiner filtering. To make a pure-B weiner filter, we represent the noise covariance matrix as the noise covariance plus the E component of the signal covariance (Bunn & Wandelt 2017), so that the filter treats any E modes as noise. This means the weiner filter equation becomes:<br />
<br />
<data_wf = S_b/(S_b+S_e+N)*data><br />
<br />
The difficulty in weiner filtering for CMB analysis lies in the fact that we can only easily write down S in the sphereical harmonic basis, and N in the pixel basis. In fact, S is diagonal in spherical harmonic space, and N is diagonal in pixel space. Because of this, we must use a work around in order to weiner filter maps. The general approach of the messenger method is to introduce a new field, t, which is called the "messenger field". This field has covariance T, which is proportional to the identity matrix and represents the homogeneous portion of the noise covariance. Being proportional to the identity matrix, T can be written diagonally in any basis we choose (Elsner and Wandelt 2012).<br />
<br />
Since we have multiple matrices that can only be written in one basis, this suggests some sort of iterative algorithm to solve for a weiner filtered map where we use the messenger field t and its covariance matrix to mediate between pixel and sphereical harmonic space. Elsner and Wandelt lay out this procedure in their paper and they come up with two equations to be solved iteratively for a weiner filtered map. In their equations, there is a lambda parameter which is a scale factor used to artificially increase the covariance T. This is useful in speeding up convergence of the algorithm. At the end of the algorithm, lambda must be reduced to 1 in order to solve for the true weiner filtered map.<br />
<br />
In our implementation, lambda begins at a value of 1300, then on the second iteration it is decreased to 100, and from then on out, lambda -> lambda*eta on each iteration, where eta = 0.825. When lambda reaches 1, our algorithm iterates another 5 times. We have experimented fairly extensively with other cooling schedules for lambda and found that our method gives the best convergence in the least amount of time. We are able to filter a single set of Q and U maps in 34 minutes on a single processor. Shown below is an example of a filtered and unfiltered map for visualization of what our algorithm is doing. The input maps below came from CMBS4 combined simulations at 95 GHz using experiment configuration 04.00. So far, the only maps we have done any work with are at 95 GHz and in this experiment configuration. The algorithm is set up to work with any set of Q and U maps, however, and thus can easily be used at any frequency with any experimental configuration.<br />
<br />
<br />
***Put in images of input and output Q maps.***<br />
<br />
<br />
The algorithm takes in a set of two maps, corresponding to a Q and U measurement, and outputs a B only weiner filtered set of Q and U maps. One can visually see that although the input map has a defined edge, the algorithm extends the output map into the unobserved region. This is because the code is coming up with a full sky weiner filtered map and thus any power detected at low ell will result in map fluctuations at large angular scales. It is also visually apparent that the input Q map contains mostly E modes (fluctuations that appear straight up and down or straight across), while the output Q map looks like it's all B modes (diagonal fluctuations). This is exactly what we would hope to see from a pure-B estimator.<br />
<br />
There is still some noise bias in the output map, however we can remove that through using monte carlo methods. There is also a suppression factor involved in the output data which is the reason for the scale difference in plots. This factor is corrected for through the use of band power window functions. Both the noise debiasing and suppression factor correction are made at the power spectrum level only. We are not capable of applying noise and suppression factor corrections to maps.<br />
<br />
Shown below is a plot of 100 spectra which are cleaned signal plus noise maps at 95 GHz taken from the 04.00 CMBS4 experiment configuration. The bins used in this analysis began at ell of 20, and went up through ell of 370 with a bin size of 35. So, bin number zero is ell of 20 - 55. Bin 9 (last bin) is ell of 335 - 370. Also included is the mean of these 100 spectra and the expectation value which was calculated using band power window functions and theory BB spectra.<br />
<br />
<br />
***Put in plots of filtered signal plus noise band powers***<br />
<br />
<br />
As one can see, the expectation value for our estimator runs directly through the middle of our simulations and is very close to our mean. It is worth noting that there is definitely something out of the ordinary happening in our last bin. In this bin, we routinely get results that do not agree with the rest of the data. Because of this, we generally ignore results from the last bin given that we are not too worried with what is happening at that ell range anyway. Shown below are plots of BB to BB and EE to BB band power window functions for both the messenger method estimator and the S2hat estimator. Band power window functions for the messenger estimator were calculated with an input signal C_ell = 1 on the sky. This means that the actual input power for deriving band power window functions is 1 * B_ell squared, where B_ell squared contains the effect of the beam. We also ran 6 simulations at each ell with this input power and then took the mean of the output across these six simulations to get the final band power window functions. Summing the window functions across ell values gives us the total suppression factor.<br />
<br />
<br />
***Put in plots of s2hat and messenger band power window functions***<br />
<br />
<br />
Again, it is visually apparent that there is something odd going on in the last bin for the messenger method estimator.<br />
<br />
The main achievement of our implementation is that we have found a lower variance in binned BB power spectra than in pure-S2hat filtered spectra. Below are plots the variance in each bin across the 100 realizations which were filtered. The exact same simulations were used as input to the two methods compared (signal plus noise simulations at 95 GHz using the 04.00 experiment configuration).<br />
<br />
<br />
***Put in plots of variances***</div>Michaelrayhttps://cmb-s4.uchicago.edu/wiki/index.php?title=PureB_by_Messenger_Method&diff=9205PureB by Messenger Method2019-09-05T14:42:17Z<p>Michaelray: Created page with "''Michael Ray, 2019-09-xx'' ---- Dr. Bischoff and myself have been working on a new approach to a pure-B estimator for CMB polarization data. The method was originally propos..."</p>
<hr />
<div>''Michael Ray, 2019-09-xx''<br />
----<br />
<br />
Dr. Bischoff and myself have been working on a new approach to a pure-B estimator for CMB polarization data. The method was originally proposed in 2012 (Elsner & Wandelt 2012) and has been dubbed the "messenger method" approach to weiner filtering. To make a pure-B weiner filter, we represent the noise covariance matrix as the noise covariance plus the E component of the signal covariance (Bunn & Wandelt 2017), so that the filter treats any E modes as noise. This means the weiner filter equation becomes:</div>Michaelray