Abstract

A method for implementing evidence-based guidance for a data science team using generative artificial intelligence under uncertainty is provided. The method includes reviewing existing evidence on generative AI in knowledge work, including empirical literature on productivity effects, cognition and skill formation, and human factors research on automation complacency and cognitive offloading. The method includes evaluating the existing evidence across parallel considerations and deriving a plurality of principles from the existing evidence. The principles include an accountability principle stating that accountability for work cannot be self-certified, an aggregation principle directing to read more but not think less, a writing reservation principle instructing to write things that require understanding, an automation principle directing to automate writing that is pure overhead but to run a test at a team level, an iteration principle instructing to iterate more and faster, and a fallibilist principle stating that all preceding principles are probably wrong in some ways. The method includes applying the plurality of principles to data science team workflows and evaluating the plurality of principles against team outcomes comprising good outcomes for clients, protection of a pipeline of future data scientists, and avoidance of de-skilling experienced practitioners.

Creative Commons License

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.

Share

COinS