The VBO bind operation is often synchronous, and needs to be waited
on by the ioctl thread. Allocate the completion struct used to synchronize
between the ioctl and bind operation on the heap for simplicity.

Change-Id: I709d417dbd3fb0ecd1150439f598fc3629de378e
Signed-off-by: Lynus Vaz <quic_lvaz@quicinc.com>

Consider below sequence from userspace:
1. Set minimum power level to 0
2. Force minimum power level constraint from a context
3. Submit commands from that context
=> _adjust_pwrlevel() returns 0 as min and max pwrlevels are
forced to 0. This sets pwrc_old->hint.pwrlevel.level to 0.
4. Set minimum power level to default value
5. Before the first constraint expires, force maximum power
level constraint from another context
=> Since constraint type is not none and pwrc_old->hint.pwrlevel.level
is same as return value from _adjust_pwrlevel() i.e., 0, power level
update doesn't happen.

Long story short, there exist races when both power level related
sysfs and constraints are modified.

To address these races, update the condition to set a new constraint
if the requested constraint is max and active constraint is min.
Update the ownership and timestamp always if old constraint is same
as requested constraint to avoid pre-mature constraint removal.

Change-Id: Id2b501fe714c51c4a5a511f88fb3ae0d244f3db6
Signed-off-by: Kamal Agrawal <quic_kamaagra@quicinc.com>

GPU VAs are allocated by looking for the first fit. In some allocation
patterns we might spend time looking for a hole. Instead, store a hint
based on the last allocated gpuaddr and use this to improve the search
time. Since this may result in VA space fragmentation do not use this hint
if limited to a 32-bit address space, or on global and secure pagetables.

CRs-Fixed: 3618766

Change-Id: Ib17bc300f53e2eaae578872a8665e5c36ae30fcf
Signed-off-by: Lynus Vaz <quic_lvaz@quicinc.com>

Update the GBIF OT size to the recommended value for Gen7_14_0.

Change-Id: I7a95c5fb1ab32170300fa473cc35209fb1e5550c
Signed-off-by: Akhil P Oommen <quic_akhilpo@quicinc.com>

KGSL internally has a limit on the length of the list of syncpoints
submitted in a single AUX command. Enforce this limit so we don't
overwrite memory beyond the structures that track these syncpoints.

Change-Id: I261bfd4f786ff7e4fbe07e8bca9e9b8d8b87c950
Signed-off-by: Lynus Vaz <quic_lvaz@quicinc.com>

When setting svm region during the gpuobj import ioctl call for a usermem
address, there is a possibility of a very large input size causing the
region's 64-bit end address to wrap around. This can cause the region
to incorrectly be considered valid, ultimately allowing a use after free
scenario. To prevent this, detect the occurrence of a wrap and reject the
import.

Change-Id: I4a88f56c58b830d4342e47dc1d1f6290c78ab6b4
Signed-off-by: Mohammed Mirza Mandayappurath Manzoor <quic_mmandaya@quicinc.com>

Add handlers for dev_pm_ops and handle secure memory
during hibernation. Load pdc, rsc ucode and re-initialize
TZ during restore. Use CONFIG_QCOM_KGSL_HIBERNATION
to enable hibernation in kgsl.

Change-Id: Ie6a519df40deb66d1fbe9ab1ef64e245defb4473
Signed-off-by: Jeyaprabu J <quic_jeyaprab@quicinc.com>
Signed-off-by: Amit Kushwaha <quic_amitkush@quicinc.com>
Signed-off-by: Abhishek Barman <quic_abarman@quicinc.com>

Add TPL1 block and update precrashdumper registers in gen7_14_0
snapshot.

Change-Id: Ia07f79192f8ffc3039b1b583c063c61ac39f43c7
Signed-off-by: Kaushal Sanadhya <quic_ksanadhy@quicinc.com>

Add ACD feature support for Crow GPU.

Change-Id: Ifdbf07b7c751511af94477f1d9783519414a17eb
Signed-off-by: Kaushal Sanadhya <quic_ksanadhy@quicinc.com>

Make avoid_ddr_stall flag target specific and convert it to
'fast_bus_hint' flag. Disable it on Gen7_14_0 gpu as it is not
recommended.

Change-Id: Ied9ce988e2058b2631c80b4a66964a6a88acc6b1
Signed-off-by: Akhil P Oommen <quic_akhilpo@quicinc.com>
Signed-off-by: Kaushal Sanadhya <quic_ksanadhy@quicinc.com>

Recently, there was a fix for a gpu bw starvation issue in a gen7 gpu.
This fix increases the GPU IB vote when gpu is at the lowest opp corner
and when it detects a high 'ram wait' above a particular threshold.

Some GPUs are less capable in terms of ddr channels, system cache,
smmu tbu size etc. So it is natural to see a much higher 'ram waits'
on these gpus. Due to this, there is now a very high IB vote in some
a6xx GPUs causing severe power regression in some use cases.

So, lets keep this ddr stall fix enabled only on Gen7 GPUs for now. We
have not seen a similar ddr starvation issue in a6xx/a5xx/a3xx family.

Change-Id: Ic89a9a90665566749e7715446ab16fdcbadb37d6
Signed-off-by: Akhil P Oommen <quic_akhilpo@quicinc.com>

Currently, we generate bus modifier hint (to update bus level) in gpubw
governor to identify the bus modifier value. Also, we leak
"wait_active_percent" stat from gpubw governor to kgsl_bus.c to again
update the bus level, something which a bus hint could have done. To make
it cleaner, we can keep the entire logic of bus hint generation logic
within the gpubw governor and pass a new hint to increase the buslevel
twice.

And refactor the bus hint generation logic in gpubw governer to make it
a little bit more readable.

Change-Id: Id996c31f20efd6735b7f79da642bd3db1aa3184e
Signed-off-by: Akhil P Oommen <quic_akhilpo@quicinc.com>

Few of the devfreq flag related code are vestigial from older kernel
versions. There is no way these code paths are exercised in the current
kernel version. Remove this along with some unnecessary conditional
compilations which always test as true.

Change-Id: Ib2bc26b3710afda92943274a85a75347799462fc
Signed-off-by: Akhil P Oommen <quic_akhilpo@quicinc.com>

Print the cmdline string of the culprit process during snapshot to
identify the exact command to trigger the same gpu hang. This will help
to make it easier to reproduce the gpu issue in scenarios involving
cmdline applications.

Change-Id: I26baa671aa7a5465cfc88cebe08aeee550e22922
Signed-off-by: Akhil P Oommen <quic_akhilpo@quicinc.com>

Gen7 crash dumper script is broken down into multiple chunks
and script will be invoked multiple times to capture snapshot
of different sections of GPU. If crashdumper fails once, chances
are it will fail subsequently as well. Hence fallback to legacy
snapshot path in case any point crashdumper script timeouts.

Change-Id: I3a4ad44cd2b354621e865e476c3d6ab64d3a7190
Signed-off-by: Hareesh Gundu <quic_hareeshg@quicinc.com>

As per the latest recommendation, we should allow write alloc for
outer-cache for gen7_14_0. This help LLC to see cacheable write traffic
from gpu. So do not set KGSL_MMU_FORCE_LLCC_NWA flag for gen7_14_0.

Change-Id: I002577ca0612f4e36ce329980e581d5042b4c276
Signed-off-by: Akhil P Oommen <quic_akhilpo@quicinc.com>
Signed-off-by: Kaushal Sanadhya <quic_ksanadhy@quicinc.com>

Set the global LLCC allocation policy to write allocate unless
the mmu feature for no-write-allocate is set for the target.
Force no write allocate for A3x, A5x, A6x and all Gen7 targets.

Change-Id: I72222872728c390932601f0fdf96c30d16deba01
Signed-off-by: Sushmita Susheelendra <quic_ssusheel@quicinc.com>
Signed-off-by: Kaushal Sanadhya <quic_ksanadhy@quicinc.com>

Enable Battery Current Limiting feature for gen7_14_0 GPU.
BCL prevents UVLO (Under Voltage Lock out) due to sudden
voltage droop and OCP (Over Current Protection).

Change-Id: I1167c4fafde3ce56d11c6f6161fafe6a49075368
Signed-off-by: Kaushal Sanadhya <quic_ksanadhy@quicinc.com>

Gfx rail might go off while accessing GPU_RBBM_PERFCTR_CNTL register during
perfcounter put which can lead to fence errors. Thus, avoid writing to this
register while releasing a countable from the counter resource.

Change-Id: Ib541cd87118b927a51a327687a8ac1db9e2f197f
Signed-off-by: Kaushal Sanadhya <quic_ksanadhy@quicinc.com>

Add new blocks and registers for the Gen7_14_0.

Change-Id: I0bfcb8ba17b46f17d95196bc0e73a5a764f855e1
Signed-off-by: Kaushal Sanadhya <quic_ksanadhy@quicinc.com>