0

I have a query where I am getting the distance for each point in my table (fields longitude and latitude) from a centroid using ST_Point and ST_DistanceSphere (below). If I want to filter the results to just distances less than 1 mile using a WHERE clause, I get the error Missing data for not-null field. If I add the non-filtered results to a temp table and then filter the temp table with a WHERE clause, it works.

Here is the query:

with sample as (
    select
        *,
        ST_Point(longitude::float, latitude::float) as coord
    from my_db.orders
    where longitude <> '' and latitude <> ''
    order by placed_at desc
), distances as (
    select
        *,
        ST_DistanceSphere(ST_Point(centroid_long, centroid_lat), coord) as distance
    from sample
    where coord is not null
)
select *
from distances
where distance < 1609.34

Here is the execution plan for the filter:

((st_distancesphere('redacted'::geometry, st_point(float8((longitude)::text), float8((latitude)::text))) < 1609.34::double precision) AND (st_point(float8((longitude)::text), float8((latitude)::text)) IS NOT NULL) AND ((longitude)::text <> ''::text) AND ((latitude)::text <> ''::text))

I do not have any not null fields as far as I can tell. All the distances are non-null float values and the underlying fields are all nullable.

Is this an issue in the execution plan and if so, how do I force Redshift to filter this correctly without using a temp table?

0