This page started as a lab meeting discussion of how to choose among different options for scientific tools (mostly focused on programming).
Question Framing
- When should I learn python/R/Stan/etc.?
- How does {tool x} integrate into my workflow?
- If I don’t learn {tool x}, what am I doing instead?
- What are the benefits for knowing how to use {tool x}
- What are the benefits for learning how to use {tool x}
Identify Alternatives
- some possible scenarios
- I have a specific research problem that doesn’t appear to solvable with my current toolset
- Lots of people are using {tool x}, and I don’t want to be left behind
- No one else is using {tool x}, and I want to be the leading expert in my lab/department/field
- what are some other considerations beyond solving my specific problem
- speed
- generality
- how do I find out about other options?
- google +3
- twitter +3
- literature search
- seminar talks, conferences (people tend to be a little more “in the weeds”, more receptive for engagement & dialogue)
- workshops
- what are the people around me using?
- peers, colleagues, collaborators
- similar enough level for learning/teaching
Evaluate the utility of knowing a new tool
Ok, @cmMcConnaughy is right - enough language war. My language-agnostic concluding thoughts. Choose tools that:
— Thomas J. Leeper (@thosjleeper) February 4, 2019
1) Have strong communities around them
2) Are proven but adaptive, so you can use them long-term
3) Facilitate persistent reproducibility, esp. w/ your future self
properties of communities to consider:
- communities to avoid (non-inclusive, wrong level)
- asking about what communities are set up around tools
- how diverse is the steering committee for the tool
- what are the practices of the community?
- If you are going to spend time in a community, will you be able to grow and take on leadership roles? (is that important for you)
What is the long-term trajectory of a tool:
- is it actively being developed?
- established history of developers
- support for older versions
Evaluate the utility of learning a new tool
- humans are habit-forming
- defer learning new tools –> (decades later) “I’m too old to learn {tool x}”
- always learning new tools –> never get around to using anything in practice
- exposure to new communities builds your professional network
- exposure to new ideas gives added context to what you already knew before
- “learning tools” is also a skill! (which means you can practice it and get better)
- “if you don’t use it, you lose it” -
Things to keep in mind
Decision making is difficult (but it is a skill you can practice and improve on!)
imperfect knowledge
- no information source is incomplete
- information sources aren’t completely tuned to your individual situation
outcomes are imperfectly measured
- how useful was learning {tool x}? (no ideal way to measure this)
- no measurement of what would have occurred with an alternative
limited time/attention
- spending (more) effort to improve decisions can also be suboptimal
when to abandon bad decisions (sunk cost fallacy)
Tips
- find good heuristics
- be aware of biases (cognitive and otherwise)
- reflect on past decisions (successful and unsuccessful)
- engage in deliberate practice
- identify appropriate goals
- evaluate outcomes
- get targeted feedback
- repeat
- embrace failure
- “People don’t fail, processes do”
- debug your processes (learn from mistakes, aim for long-term growth)
- learn to fail fast
- for large/complex projects - is {tool x} suitable long-term?
- test whether the {tool x} addresses the most difficult aspect of the project
- if this doesn’t work, it is better to know sooner!
- exposure vs. learning
- goal-oriented or not?