Difference between revisions of "Technology Development Telecon - Combined"

From CMB-S4 wiki
Jump to navigationJump to search
Line 38: Line 38:
 
** Announcements - github for PDF: https://github.com/CMB-S4/Memos/tree/master/TDEvaluation
 
** Announcements - github for PDF: https://github.com/CMB-S4/Memos/tree/master/TDEvaluation
 
** Review charge
 
** Review charge
 +
** ICCC request for intermediate product to use for DOE budget process.
 +
***Can we use/modify CDT's broad priority list for TD?
 +
***Background material: [[File:2018-01-27_CDT_R%26D_for_TD_group.pdf]]
 
** Go over our progress, discussion we had, how we are doing
 
** Go over our progress, discussion we had, how we are doing
 
** Discuss goal by ANL S4 meeting
 
** Discuss goal by ANL S4 meeting

Revision as of 00:41, 25 January 2018

Charge to CMB-S4 Technology Development Working Group

In the first edition of the Technology Book, the experimental CMB community summarized the current state of CMB technology and evaluated its current technical readiness with a 5-level Technology Status Level (TSL) and manufacturing readiness with a 5-level Production Status Level (PSL). For each technology, we identified Technology Development (TD) efforts necessary to advance it for possible use in CMB-S4. As a next step of the collaborative community wide effort, the CMB-S4 TD prioritization working group will evaluate TD topics based on impacts they have on cost, schedule, and science return. By the time of the Argonne meeting (March 2018) the working group will produce a prioritized list of the TD topics that the community should pursue to ensure timely maturity of technologies that will enable the successful advancement of the project.

We have grouped the relevant technologies into the following areas to tackle this immense task: Telescope and Site; Cryogenics, Cryostats and Optics; Detectors and Readout; and Data Management. Calibration of evaluation metrics across the subgroups is important for fair comparison of the TD topics. In addition, many TD topics are inter-dependent. To capture these ideas, the overall working group will communicate across all the subgroups in monthly combined group meetings.

The scope of each subgroup is as follows:

  • Telescope and site: Covers telescope, mount, site, power generation, etc...
  • Cryogenics, cryostats and optics: Covers cryogenics (4K and mK), cryostats, windows, filters, lenses, HWP etc...
  • Detector and readout: Covers detector (detector array and holder) and readout (warm/cold), etc...
  • Data management: Covers DAQ, data transfer, simulation, analysis, publication, etc...

Tab-Separated Table

https://docs.google.com/spreadsheets/d/101ncyzfDAHrTF9O0rTPRGbX6dqut_WdfVA7-2ck9qGQ/edit?usp=sharing

Milestone

  • October (combined call on Nov 2)
    • Define baseline to compare impact of TDs to
    • Start populating list of TDs item and do evaluation on some entries
    • Make modification/improvements to organization method if necessary
  • November (combined call on Nov 30)
    • Continue to populate and evaluate list of R&D items as sub-group
    • Draft list by end of month
    • evaluation may not be complete by this time
  • December (combined call on Jan 04)
    • Sub-group list with priority evaluation by end of month such that we can start combining lists in 2018
  • January (combined call on Feb 1, maybe more combined call)
    • Start normalization/ combine lists from different groups
    • Draft of combined list by end of month
  • February (combined call on March 1, maybe more combined call)
    • Modify/fine tune combined list
    • Discuss on what we’ll show at CMB-S4 workshop at ANL
  • March
    • CMB-S4 workshop at ANL: Present at CMB-S4 meeting

CMB-S4 Technology Development Telecon 2018-01-25: Combined Call

CMB-S4 Technology Development Telecon 2018-01-04: Combined Call

  • Agenda
  • Note
    • Organizing PDF document for final report:
      • github will be setup such that github repo will be in same location as the first technology of the technology book. TD evaluation will be placed in new folder. You will be able to use same pull request to get the document. Please contact Tom Crawford if you do not have github access for CMB-S4 documents.
    • Inputs from sub-groups:
      • Common thread from different sub-groups
        • What worked for S3 may not work for S4 due to production rate
        • It was difficult to fill out money related entries such as expected cost reduction and estimated investment
        • Adding "baseline" column right next to TD entry was useful
        • CDT was helpful, but only had limited assumptions
          • CDT should be used as a configuration that works and not as definitive experiment design that we'll deploy
          • Steve Padin collected many quotes & estimates from experts to make cost estimate. He would be helpful resource to fill out cost/investment related entry. We cannot share his spreadsheet because it contains proprietary quotes.
          • Using other sources (for example Simons Observatory study)to define baseline is also useful. Enter assumptions in the spreadsheet.
        • Low participation
          • Suggestions: Use meeting time to identify experts on an entry. Then ask expert to contribute few lines instead of entire entries.
          • Send encouraging email from ICCC to encourage engagement from CMB scientists (Julian)
          • Please encourage your friends to call in!
        • Split technology development by large/small telescope and southpole/chile if different size of telescopes and/or sites require different TD
        • Would like to add author and citation column would be useful
        • Technology book is a useful resource to learn about capability/current status of technologies
        • CMB-S4 forecast/inflation group, Simulation and Forecasting Logbook, can give feedback on science impact of technology - Contact: Lloyd, Kovac, et al.
        • Schedule driven priority (DM group is doing this for ex.) is useful
      • Presentation at CMB-S4 ANL meeting
        • We should be proactive about what we would like to present
        • Realistic goal? - Complete sub-group evaluation by March meeting

CMB-S4 Technology Development Telecon 2017-11-30: Combined Call

  • Agenda
    • Slide - File:2017 11 30 CMBS4 TDCombinedNovember.pdf (Toki)
      • Reminder/adjustment about schedule (please see mile stone above)
      • Technology development table baseline - CDT and beyond: how each sub-groups are dealing with baseline that were not defined in CDT report
      • How to define/criteria ("1-5" etc) in technology development table
    • Report from sub-groups would be helpful
      • Detector-Readout update: File:20171130 TD DR update.pdf
      • Maria's Input
        • Assuming Stage-III experiments experience, technology level as fundamental reference;
        • Not baselining something which has low TRL and that would require a strong R&D (timeline and money), for example not Sapphire HWPs;
        • Some assumed TD items which has worked well up to Stage-III experiments (so point 1.) might be not be the same for CMB-S4 just because of the scale of the project. For example: filters provided by Cardiff (1filter/2 weeks).
  • Notes
    • CDT report made assumptions that are not written out in the report to come up with final cost estimate. Toki is pinging CDT member to see if this information can be released. Zeesh and Toki could answer some specific question as both have access to preliminary version of the cost worksheet.
    • Detetcor/Readout team followed schedule rating (1-5) from the worksheet that Zeesh and Brad made. Action item on Zeesh to circulate schedule rating metric
    • Schedule saving are to be used for saving on construction time
    • Science statics saving should be saving on mapping speed. This way we do not mix operation time and construction time. Cleaner from DOE scheduling point of view
      • 1 = 0%, 2 = 5%, 3 = 10%, 4 = 25%, 5 = 50%
    • Science systematics - There are theoretically/analytically known systematics that we need to be ready to tackle on (for ex. band pass measurement) but hardware aren't ready.
      • Add wording in metric to put more value into reducing bigger systematics uncertainty
    • Swap order of column such that Likelihood of success comes at the end of TD
    • There was interest to reinstate risk mitigation/remaining risk framework. Maybe add appropriate column at end of the table? Zeesh to present idea at next meeting
    • Integrated test is TD topic we need to keep eye on. Let all sub-groups cover. We will deal with overlapping coverage when we merge.
    • We dropped TDM from readout for S4, but if we are baselining to use existing technology, TDM works. It is tedious to assemble. Add TDM back in? Quantify assembly work.
    • Toki to send out updated evaluation metric
    • Toki will setup .tex in github - Do this before we start combining tables in Jan 2018

CMB-S4 Technology Development Telecon 2017-11-02: Combined Call

  • Notes
    • Toki to github for .tex
    • People were not sure about how to define baseline experiment.
      • Great suggestions made in CCO presentation's last slide #5
      • Read and extract as much baseline model as we can from the CDT report.
      • For detailed baseline that is not explicitly define in the CDT, each sub-group will discuss baseline they need to add to work through the worksheet.
      • But then we want to make sure assumptions are same. So we added new tab call "baseline" where we collect assumptions we are using. Sub-group leads will
      • We also added new tab call "expert contact" just so that we know who to call up when we need expert's advice on how to evaluate
    • We are still flexible to modify columns. Let's start evaluating TD topics with basline extracted from CDT. On Nov 30th call (or before that through email) we will discuss what worked, and we will converge on column entries to use across different sub-groups

CMB-S4 Technology Development Telecon 2017-10-12: Combined Call

  • Agenda
  • Notes
    • Reports from sub-groups
      • CO (Maria): first group meeting is scheduled for later today.
      • DR (Tom): ongoing discussion of prioritization method during first meeting
      • DM (Julian): Walked through DM portion of CDT report and provided feedback to update report. Plan to start with areas of concern and work towards prioritization and a nominal schedule
      • TS (Brad): Brad has agreed to serve as co-lead and has spoken with Akito.
    • Reviewed slides covering sub-groups, working group output, prioritization method, and milestones
      • Discussion on prioritization method
        • Sense that the sub-groups should have a common reference point when prioritizing to ease normalization process
        • Likely to start simple, looking at TD topics, and move towards more detailed prioritization considering schedule and risk.
        • Discussion of need for some sort of baseline. This could be useful is assigning metrics, but is not necessarily part of this exercise
    • Repost of links to prioritization methods
    • Reminder that we are discussing Technology Development (TD) and not R&D (so don't use it)
  • Action Items
    • attend CDT report telecon on 10/13/17 at 11AM Pacific (everyone strongly encouraged)
    • create pages on the wiki for each sub-group (Toki)
    • add sub-group pages to TD template google doc (Toki)
    • sub-group leads will discuss charge, prioritization methods, and schedule with groups and report comments back to Toki and Zeesh

CMB-S4 Technology Development Telecon 2017-09-28: Combined Call

  • Technology Development section in CDT report?
  • Group organization - Pick organizer
    • Combined call every 4 weeks
    • Telescope and site: Covers telescope, mount, site, power generation, etc...
    • Cryogenics, cryostats and optics: Covers cryogenics (4K and mK), cryostats, windows, filters, lenses, HWP etc..
    • Detector and readout: Covers detector (wafer and holder) and readout (warm/cold), etc...
    • Data management: Covers DAQ, data transfer, simulation, analysis, etc...
  • Ideas on how to organize and evaluate TD topics
  • Note
    • TD section in CDT report
      • Still in works
      • Listed categories are: Detector module, optical coupling, readout, focal plane, receiver, optics, pol mod, cryogenic, monitoring calibration validation, simulation (still need to be added)
      • Should topics in CDT be in line with 4 categories below? > It pretty much already is that way
    • Sub-group organization
      • Sub-section leads
        • Combined call: Toki (every 4th week for now. Will adjust frequency as necessary)
        • Detector and Readout: Tom (ANL), Patty (Princeton), Shawn (SLAC)
        • Data Management: Julian (LBNL), Joy (USC), Laura (Yale)
        • Cryogenics, Cryostats, Optics: Abby (Chicago), Maria (Paris)
        • Telescope and site: Come up with nominations. Suggested Akito. Experts from South Pole site would be nice as well
      • Meeting Organization
        • Toki and Zeesh to write charge and scope for each sub-groups
        • Detector team will use Thursday 9 AM (pacific) = 12 PM (eastern) slot. Please use same telecon call in as well.
        • Other groups have to find other meeting time. Unfortunately we maybe be forced to live with overlapped meeting.
        • Sub-group leads create doodle poll that matches with their schedule. Send doodle link to Toki such that we can send out one centralized email
        • Sub-group meeting next week, combined call week after
    • Organization Method
        • Risk based and alternative TD based organization method was presented
        • Both uses very similar idea. TD based organization is basically "opportunity tab" of risk based method. Why have two separate tabs for risk and opportunity for risk based method? How will topics in risk be evaluated against opportunity were brought up. TD method tries to get around that by providing flat structure that bases no TD topic
        • Pointed out that both method are necessary to do this right, but not creating too much
        • Risk assessment assumes some baseline which we could use CDT report as a base. Success of LSST could be attributed to maturity of risk/opportunity assessment matrix they had. Question of if this is right model to use at this stage of CMB-S4 when there is no solid baseline yet was brought up.
        • Sub-group to discuss to see if one model fits better (or need both), or come up with alternative way to organize TD topics. We will discuss on combined call week after next.
        • Zeesh/Brad/Nadine to post their example on risk based approach
  • Action Items
    • Contact Akito for Telescope and site lead. Come up with South Pole site/telescope expert to co-lead (also Chile if Akito cannot)
    • Toki and Zeesh to write charge and scope for each sub-groups
    • Sub-group leads to communicate, setup doodle poll for the group, send Toki a doodle link, Toki will send out centralized doodle email
    • Keep Thursday 9 AM pacific = 12 PM eastern open for detector/readout or combined call
    • Sub-group call next week (Oct 5 for detector and readout, other groups TBD), combined call week after (Oct 12)
    • Sub-group to discuss how to organize, think about best way to organize TD topics
    • Zeesh/Brad/Nadine to post their example on risk based approach

CMB-S4 Technology Development Telecon 2017-09-21: Combined Call

  • Meeting time
    • Thursdays 9 AM ~ 10 AM (pacific), 12 PM ~ 1 PM (eastern)
    • Meeting frequency: Weekly
  • Output
    • Create opportunity, impact, risk assessment matrix
      • Matrix to organize impact TD would have on science, schedule, and cost with likelihood of its occurrence
    • Relatively short document (compare to the technology book) that describes TD priority for the CMB-S4
      • (Toki's view) We can work on excel sheets, but we should put together a written document
    • Timeline - Next S4 meeting is in 6 month. March 5 - 9 (doodle! https://doodle.com/poll/r77rgs67fracgi6f)
    • (Toki's view) Internal draft/basic idea/ agreement on material by then - Polishing will probably take a bit longer
  • Organization
    • Form sub-group to go into categories in detail - Pick a person to organize sub-group telecon
    • Combined call once a month (approximately every 4th call) to coordinate/ cross-calibrate across groups
    • Let's have another combined call next week before we split into sub-groups
  • Categories to cover
    • Telescope - This also includes telescope control and monitoring software
    • Optics - Lenses, HWP, Filter etc....
    • Detector - Fabrication and validation
    • Readout - Warm and cold electronics
    • Cryogenics - to 4K, and sub-Kelvin, cryostat design, vacuum window
    • Data acquisition for bolometer data, thermometry, house keeping
    • Simulation/Analysis/DAQ assessment
      • Should this be included in instrument heavy TD plan? Or should it be covered by Sim/Comp working group separately?
    • Other categories?
  • Notes from the Meeting
    • On output of this study
      • We will work on this using excel spread sheet. Report from this study should be written document. It will be also helpful to prepare power point presentation for people to use in talks
      • This will be evolving document
    • On Organization and topics to cover
      • Split into 4 groups - We can do four groups! We did it the first edition of the technology book. Groups are free to divide further into subsub groups. We'll decide on group organizer at next call
        • 1) Telescope and site: Covers telescope, mount, site, power generation, etc...
        • 2) Cryogenics, cryostats and optics: Covers cryogenics (4K and mK), cryostats, windows, filters, lenses, HWP etc..
        • 3) Detector and readout: Covers detector (wafer and holder) and readout (warm/cold), etc...
        • 4) Data management: Covers DAQ, data transfer, simulation, analysis, etc...
        • Please think if we covered TD topics well. Hopefully we can insert additional topics into these four groups.
      • Normalizing is important to make sure risk in group 1 is evaluated with same metric as in group 2
      • We'll have 1 combined all group meeting for every 3 sub-group meetings (assuming 1 meeting per week)
    • On how to evaluate TD
      • Considering timing is important. There are different priority for pre-CD0, CD0, CD1, and CD2
      • For now, we should focus on TD for pre-CD0, during CD0
      • Concern for not capturing dependencies between different TD items were pointed out.
      • Approach that LSST took was shared
        • How to evaluate 'risk' when baseline is not well set - Related concern was raised during S4 meeting
        • Table that's complicated may prevent people to get engaged
        • For a DD with evolving risk, we can separately write down specific risk/mitigation with a date for milestone
        • Some risk could have multiple mitigation method - Simply add more rows for mitigation
    • Action item for next meeting - We will have combined call again in 1 week.
      • Ideas on how to organize and evaluate TDs were posted. Also examples from LSST were shown. Please review and think about what is the best way to organize and evaluate TD for CMB-S4
      • Also something to think about:
        • How to evaluate 'risk' when baseline is not well set - Related concern was raised during S4 meeting
        • Table that's complicated may prevent people to get engaged
        • Concern for not capturing dependencies between different TD items were pointed out.