Proposal for Mobile Device Working Group

[OBWANDO]

Proposal for Mobile Device Working Group

• WG (Working Group) is something new to O3DE. Typical relationships between SIG and WG can be:
  • with SIG expected to be long running oriented to a particular area of interest;
  • with WG expected to be short (relative to SIG) running with clear need of cross-cutting processes (collaborations) across SIGs
• May want to consider “Mobile WG” (current name) or “Mobile Device WG” (“device” will highlight the hardware specific need or focus, if it turns out to be the case.)
• “Mobile” meaning phone/tablet/HMD, excluding watch/wearable?

Overview of WG

With a general vision enabling O3DE community to build high fidelity and high efficiency mobile games, this WG focuses on running O3DE on mobile, characterized by technologies and challenges on supporting:

• Arm Based
• Mobile GPU features, e.g. Hardware accelerated Ray Tracing, VRS
• Mobile OS
• Mobile only API
• Mobile Benchmarking on new applications: Mobile Ray Tracing

Goals

Good runtime experience of running O3DE on Mobile targets.

Scope

Generalized overall scope of work is anything that impacts O3DE runtime experience on Mobile.

notes:

• any particular works identified or expected related to networking (available on Mobile)?
• any particular works identified or expected related to HMD (standalone/host connected)?

In scope

• Mobile SoC/Architecture: compatibility
• Mobile resource management (CPU cores, frequency, DDR & GPU frequency, thermal control)
• OS Compatibility: Different Android Version, OS other than Android/iOS
• TBR/TBDR support and optimization on mobile
• Unique mobile GPU feature support: VRS, RT
• Mobile only API: OpenGLES
• Lightweight rendering pipeline
• Hybrid rendering pipeline (Rasterization + RT)
• Lightweight asset pipeline
• Mobile perf tuning and profiling toolset.
• Popular Game Engine Features: ADP

Out of Scope

• (cloud gaming/game streaming?)
• (watch/wearable)

Cross-cutting Processes

• SIG-Platform on:
  • Mobile SoC/Architecture: compatibility
  • Mobile resource management (CPU cores, frequency, DDR & GPU frequency, thermal control)
  • OS Compatibility: Different Android Version, OS other than Android/iOS
• SIG-Graphics-Audio on:
  • TBR/TBDR support and optimization on mobile
  • Unique mobile GPU feature support: VRS, RT
  • Mobile only API: OpenGLES
  • Lightweight rendering pipeline
  • Hybrid rendering pipeline (Rasterization + RT)
• SIG-Content on:
  • Lightweight asset pipeline
• SIG-Testing on:
  • Mobile perf tuning and profiling toolset.
• SIG-Core on:
  • Popular Game Engine Features: ADP

[russsun]

• "“Mobile” meaning phone/tablet/HMD, excluding watch/wearable?
  Comment: Mobile should include all mobile devices forms including wearables. But the priorities can be different, since there are not too much gaming on wearables.

Goals

• Good runtime experience of running O3DE on Mobile targets.
  Comment: Good mobile developer experience as well.

Scope

• any particular works identified or expected related to networking (available on Mobile)?
  comment: there are many mobile os services to be tuned or constructed on mobile devices, cellular network tuning or mobile cpu tuning are examples. Also other platform related services like mobile payment or ads. We need to include them in the charter, but temporally lower the priority.

• any particular works identified or expected related to HMD (standalone/host connected)?
  comment: mobile WG needs to focus on any devices built with mobile architecture (ARM based SoC), no matter it is standalone (oculus quest) or tethered (huawei AR). We exclude those devices host connected to PC/Console.

In scope

• Mobile Developer friendliness in IDE and build pipeline.
• Mobile perf benchmarking tool.

[byrcolin]

Sig/Platform

• Sig vs. Working Group

We are in agreement that the permanence of a Sig is not warranted at this time, and as a matter of course a working group should always be the first instinct and promotion to Sig if warranted.

• “Mobile” meaning phone/tablet/HMD, excluding watch/wearable?

We are not sure that HMD should be called out here. Also why are watches and wearables (and HMD is a wearable) not covered? We know there was talk about a sig/XR but we chose to pass currently... We think you are saying you want a label that sets mobile HMD apart from tethered HMD: Doesn't seem unreasonable.

• Popular Game Engine Features: ADP

What is ADP? Implementing "popular game features" doesn't seem mobile related, unless you have a example?

Conclusion:

We agree in general and support the idea of creating a working group to take the lead on these issues and see if a sig is ultimately warranted. We believe the intention here is to increase mobile partner engagement and anything we can so to support that effort should be tried.

[russsun]

ADP is short for adaptive performance. It builds a communication channel between system (Android) to game engine, so when system is over heating or overloaded, the game can reduce some visual quality to maintain a smooth gameplay experience. You can find more here: https://docs.unity3d.com/Packages/com.unity.adaptiveperformance@4.0/manual/index.html
Android, Unity and ARM are quite active on this direction.

In a summary of my previous comment of defining Mobile: Mobile is about the processing hardware architecture "ARM-based" (maybe battery-powered?) instead of the device form.

[amzn-pratikpa]

Some questions that I have:

• How do we deal with code that's been submitted for other platforms that doesn't compile or is supported on mobile platforms? Can we really expect all contributors to build and test on mobile?
• How are we going to deal with mobile testing to AR?
• How are we going to test Atom across all the different devices? This is going to be especially challenging on Android and its many variants (even for the same model of device which could have different GPU's depending on the territory)

[adiblev]

It would be a bad design to have code that does not compile on some of the platforms. The feature / capability might not be fully supported as not all features are guaranteed to be supported by low end devices, but the support / lack of it should be through the RHI and not at the higher code level.
In other words - O3DE and Atom should compile on all platform but selective functionality might not be (fully) supported by selected platforms in a way that will not be a blocker.
Specific pipeline for mobile is already in place and should be kept in check exactly for that.

[jeremyong-az]

One observation I have is that several of these items in the "In Scope" list already fall under the existing SIG graphics/audio charter so we could either shuffle those responsibilities here (if it makes sense), or move some of these items out and keep them in SIG graphics/audio. If we want some form of shared ownership, we'll need to organize things so that the two groups don't inadvertently step on each others toes. This is mentioned in the "cross cutting" portion, but I'm curious if there's value in having the responsibility split across two groups, instead of consolidating.

[jeremyong-az]

Other observations, I'm not sure what would make "mobile VRS" or "mobile RT" any different than how we would need to author this abstraction for desktop. Both systems need an RHI platform abstraction and feature flags for variable support on the client device.

OpenGLES would certainly be needed for actual mobile support as a first class citizen, but this is a pretty sizable undertaking. Not that it shouldn't be done - this is just a callout.

TBDR optimizations apply outside mobile as well unless we are considering integrated GPUs as "mobile class" hardware.

[russsun]

Good call. Same abstractions definitely need to be used. Just some extra work needs to be done to adapt PC VRS to mobile VRS due to the fragmented mobile GPU market (support/unsupported and different supported sampling rates), Same for mobile RT, due to staggered stages of hardware accelerations and limited computing resource, mobile RT is always a hybrid approach for the coming years.

I agree with you on OpenGLES part.

[OBWANDO]

I'd like to set up a meeting for us to discuss some of these areas and how they may clarify where it is generic and where it is mobile. I suspect that the mobile SIG may be providing extended functionality focused on that area and the conversation can help define these areas of where the current SIG ends and this SIG begins.

[jeremyong-az]

Definitely think it's worth having a meeting with all the relevant stakeholders. The proposers and candidate chair/co-chair of the mobile device sig and graphics/audio and platforms I think would be useful to get in a (virtual) room.

[rgba16f]

Did this discussion move elsewhere?
Sig-mobile was announced at O3DCon.

[jeremyong-az]

Not that I'm aware of, this is still the correct thread to comment/discuss on the proposed charter

[lemonade-dm]

Another cross-cutting effort need to take into account is the binary size of mobile applications(APK or IPA).
It was brought up at O3DCon that the size of APK for android build is over 4GiB when using the non-monolithic workflow due to amount of shared library(.so) in the APK.

We need to take into account the mobile workflow to streamline mobile developers to make smaller binaries such as defaulting Android and iOS to default to Monolithic builds.
Other observations is to figure out whether the VFS(Virtual File System) layer within the Engine is appropriate for the development asset deployment process and whether that is a cross-cutting effort with the mobile working group. The VFS layer can stream asset data over to a device during running via an AssetProcessor running on the host platform. but the workflow for setting it up is not streamlined.

[lmbr-pip]

IMHO may want to add cross-cutting tasks on:

• SIG/Network: Mobile specific networking algorithms, config or utils
• SIG/security: Security features specific for mobile applications.

[rainbj]

IOS and Android have a set of standards required for publishing on their platforms. This includes screen sizes, recommended device speeds, inputs, patterns, etc. It would be good to know if this group will help maintain these standards (defaults?) and how they are going to be included into the editor.

[moudgils]

Agreed that sig-mobile can not be responsible for all things mobile as that will incentivize other groups (i.e cross-cutting sigs) to not consider mobile as part of their development process which may be detrimental to mobile development as a whole. This group should mostly focus on areas that other groups may not be considering given that mobile is not a first class citizen at the moment. For example, one of the most critical things we need at the moment the ability to deploy and launch to a mobile device from Editor. This capability is essential for any sort of development work on mobile as building code/data and deploying assets/apk via command line is quite a bit painful on android. Tasks like these can be overlooked by other groups and should be covered by sig-mobile group.

[andythethird]

My pre alpha project for a new launcher ui,
what do you think guys?

[andythethird]