I started my blog

This week, I started this blog to document my progress and findings as I delve into various papers, books, and other resources. I'm realizing just how vast the area of phenomenology and its philosophical framework truly is. It's amazing how you can study a person's lived experience in relation to any phenomenon. From something as simple as drinking tea or morning routines, to more complex things like work in a medical setting or traumatic experiences – it's literally quite infinite. Each person's experience is unique, and their perspective is just as valid as anyone else's.

I am on week 32 since registering as a PhD student. I’ve completed the Graduate Research Skills module (10 credits) and am now in the middle of the second module, Research Strategies (10 credits).  I am doing this part-time which really means putting approximately 15-20 hours per week.

I am situating my research in the study of the lived experience of the user’s interaction with mixed reality digital twins (MRDTs). This single sentence line alone will require unpacking as each topic area is a whole separate study in itself. However, the combination of these will help me narrow my research scope and be focused on what my research design is to be.

This might be a long post as I am just setting the context and background of where I am and where I want to be as I embark on this academic journey. The language expression in this blog will be less than formal, and I will treat this as my diary and a brain dump of sorts. It’s a privilege to do this. I get to do this!

 

What I plan to do

I want to construct my own phenomenological framework in studying MRDTs. I want to understand from the literature the various data collections that capture lived experience accounts. To initially make observations on what other authors are using and why their particular methodology is appropriate and valuable. It would probably mean a significant survey of existing studies. I need to put together a robust study plan to answer my research question.

What I already have coming into this

For context, I have been working with extended reality (XR) technologies for over 5 years at this stage. I work with a team that develops virtual reality (VR) and mixed reality (MR) applications for various applications in the education and industry space. We even conduct requirements elicitation and design workshops to inform the app design prior to development so we can fulfil and address the problem statement.

In all of this, however, I have not looked into the subjective experience of the user besides conducting usability studies which look more at the system aided by user feedback but not necessarily or purely from their point of view.

Digital Twins

Adding another layer into this, digital twin (DT) technology is enabling technology that is gaining prominence in various industries (manufacturing, healthcare, etc) (Semeraro et al., 2021). As a brief overview, DTs comprise three main components: the physical asset, its virtual counterpart and the real-time bi-directional data communication between the two. This is the most basic that I can describe in my own understanding. This means that the whole infrastructure to make this possible exists as data needs to be facilitated for this constant back-and-forth transmission. Not to mention the digitisation of sensed signals by the sensors, the data cleaning, data storage, data processing, data translation, serialisation and de-serialisation, mapping and visualisation. The journey of data is a fascinating study in itself - from sensors in the world to visualised information to the human user. It really is assigning meaning to constantly changing material properties as discretised data is relayed and interpreted. This real-time information becomes valuable as it informs decision-making towards efficiency and improved performance.

 

Mixed Reality Digital Twins (MRDTs)

Combining MR and DTs makes a unique amalgamation of technologies that expands the potential for not only experiencing data in new ways but also applying meaning in a way that was not possible before. I love seeing and interacting with holograms. Advancements in spatial computing and head-mounted displays (HMDs) have made it possible close to mainstream adoption. It is still predominantly confined to the entertainment/gaming sector, at least as perceived by the masses.

MR is the blending of virtual artefacts and the physical environment. It sits in the middle of the extended reality continuum (Milgram & Kishino, 1994). It is not fully occluded like VR and unlike augmented reality (AR), it can be interacted with thanks to the physics engine afforded by game engines. An MRDT to me then, is a digital twin, more like a component twin or an asset twin in the MR modality. These are not super new as DTs have been around as system and process twins for more than a decade now but these are presented oftentimes as dashboard interfaces – perfect for real-time alerts and simulation to improve existing processes and business models. The visuospatial context would certainly affect the user's perception and possibly enhance greater interaction afforded by the immersive nature of the environment.

What is my research then?

It is to study the lived experience of one/s who interacts with these holograms (MR) who will have some real and observable connection to the physical object (DT). My hypothesis is that it adds a complex layer and dimension to MR than simply experiencing static MR artefacts. This in a nutshell is what I want to unpack. Hence, phenomenology. I have also become fascinated and interested in this whole new world that philosophers have philosophised about for many years. I came into this cold but I have done much reading since then.

I learned that phenomenology is an absolutely massive topic. That there are branches or schools of thought within it. Philosophers themselves have built upon and/or critiqued previous proposals in the field. Edmund Husserl seems to be the one dubbed as the father of big “P” Phenomenology and from him, various thinkers have sprung out such as Martin Heidegger, Jean-Paul Sarte and many others.

They’re the ones who thought deeply about human experiences, and out of that came the qualitative framework or research approach, small “p” phenomenology, studying the structures of consciousness (Smith, 2018). These are adopted by many researchers who want to gain insights from a first-person experience of a particular phenomenon.

One key central aspect of phenomenology is intentionality –

The notion that our consciousness is always directed toward something. We are always conscious of something – whether on an object or a thought or a feeling. We don’t experience things in isolation but always in relation to ourselves.

Toward or about something, This “aboutness” implies that consciousness is always situated in relation to an object – always linked to the outside world.

Phenomenology insists that the self’s experience is shaped by its interactions with the world which is integral to how meanings are formed.

A person experiencing something, directs their attention and focus to an object, in my case, an MRDT, and so how do I retrieve some of that information? What can I do to extract data from that? What will I even expect to collect? What forms do these data take? Is it measurable, as in in an empirical way?

This is what I am currently grappling with. I hope then to develop a framework, phenomenological framework for MRDTs.

Observed research gaps

There are plenty of papers, use case studies, and development frameworks that delve into the making of MRDTs. Who is out there studying people’s experiences with them? I suspect that not many have access to this tech although it is becoming more mainstream but those who do, look more into the technical aspects of it. As I have seen so far, many assume a UI design and use existing templates. It looks to me like a translation of 2D screens into MR environments (Bogdanov, 2024; Xing et al., 2024). I understand that you need to make design decisions for the application to exist in the first place. But is it the best way? Is it appropriate?  I am not about to embark on designing UI and developing frameworks in the actual design of these MR apps but maybe as a spillover of my curiosity as I look into people's accounts, insights will come out of it that might just be beneficial. For starters, I want to make observations, and capture them in the most appropriate way. Argue why a particular method or set of methods is relevant and why they wouldn’t be.

I would need to do a survey, a sweeping broad overview of what’s out there in literature to at least point me in a direction that I can take. I can refine the paths as I go.

So where has that led me so far?

Phenomenology is too all-encompassing, it touches all of human life really. From emotional, mental and spiritual to physical and psychological – all are at play. And so I found out about Don Ihde’s postphenomenology as my supervisor, Trevor Hogan pointed me to. His human-technology relations (Ihde, 1975) fits my inquiry. MR and DTs are advanced technology unlike what humans have seen before. He has looked into how people do not just interact with technology but argues that technology is intricately linked to how we live and experience the world. Humans do not merely use technology, but technology mediates humans with their world. This fits the bill for me as MRDTs by their nature do that as most technology do I suppose.

So on to my study plan

I need to solidify a number of things:

What do I want to look at?

How do I go about it?

And be super strict on my reasons for my decisions, -  why am I choosing to use this over this?

Before looking at any papers, I need to have a robust study plan – My Study Plan

 

References

Bogdanov, V. (2024, March 17). Mixed Reality Use Cases and Challenges in 2024. Rinf.Tech. https://www.rinf.tech/mixed-reality-use-cases-and-challenges-in-2024/

Ihde, D. (1975). The Experience of Technology: Human-Machine Relations. Cultural Hermeneutics, 2(3), 267–279. https://doi.org/10.1177/019145377500200304

Milgram, P., & Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems, 77(12), 1321–1329.

Semeraro, C., Lezoche, M., Panetto, H., & Dassisti, M. (2021). Digital twin paradigm: A systematic literature review. Computers in Industry, 130, 103469. https://doi.org/10.1016/j.compind.2021.103469

Smith, D. W. (2018). Phenomenology. In E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy (Summer 2018). Metaphysics Research Lab, Stanford University. https://plato.stanford.edu/archives/sum2018/entries/phenomenology/

Xing, Y., Fahy, C., & Shell, J. (2024). Assessing web 2D user interface experiences in mixed reality. Heliyon, 10(11), e31916. https://doi.org/10.1016/j.heliyon.2024.e31916