-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[fix](meta) fix catalog replay error #35532
Conversation
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
run buildall |
TPC-H: Total hot run time: 41064 ms
|
TPC-DS: Total hot run time: 173153 ms
|
ClickBench: Total hot run time: 30.88 s
|
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
@@ -163,6 +163,11 @@ public boolean containsResource(String name) { | |||
} | |||
|
|||
public Resource getResource(String name) { | |||
// nameToResource == null iff this is in replay thread |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
iff ==>if
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not a typo, iff
means if and only if
``` Caused by: java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936) ~[?:1.8.0_301] at org.apache.doris.catalog.ResourceMgr.getResource(ResourceMgr.java:166) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.datasource.CatalogProperty.catalogResource(CatalogProperty.java:67) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.datasource.CatalogProperty.getOrDefault(CatalogProperty.java:77) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.datasource.ExternalCatalog.setDefaultPropsIfMissing(ExternalCatalog.java:173) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.datasource.hive.HMSExternalCatalog.setDefaultPropsIfMissing(HMSExternalCatalog.java:238) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.datasource.ExternalCatalog.gsonPostProcess(ExternalCatalog.java:687) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.persist.gson.GsonUtils$PostProcessTypeAdapterFactory$1.read(GsonUtils.java:640) ~[doris-fe.jar:1.2-SNAPSHOT] at com.google.gson.TypeAdapter.fromJsonTree(TypeAdapter.java:299) ~[gson-2.10.1.jar:?] at org.apache.doris.persist.gson.RuntimeTypeAdapterFactory$1.read(RuntimeTypeAdapterFactory.java:289) ~[doris-fe.jar:1.2-SNAPSHOT] ``` Introduced from apache#33610. When read meta image, the `resource` maybe null, we should ignore it.
``` Caused by: java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936) ~[?:1.8.0_301] at org.apache.doris.catalog.ResourceMgr.getResource(ResourceMgr.java:166) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.datasource.CatalogProperty.catalogResource(CatalogProperty.java:67) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.datasource.CatalogProperty.getOrDefault(CatalogProperty.java:77) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.datasource.ExternalCatalog.setDefaultPropsIfMissing(ExternalCatalog.java:173) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.datasource.hive.HMSExternalCatalog.setDefaultPropsIfMissing(HMSExternalCatalog.java:238) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.datasource.ExternalCatalog.gsonPostProcess(ExternalCatalog.java:687) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.persist.gson.GsonUtils$PostProcessTypeAdapterFactory$1.read(GsonUtils.java:640) ~[doris-fe.jar:1.2-SNAPSHOT] at com.google.gson.TypeAdapter.fromJsonTree(TypeAdapter.java:299) ~[gson-2.10.1.jar:?] at org.apache.doris.persist.gson.RuntimeTypeAdapterFactory$1.read(RuntimeTypeAdapterFactory.java:289) ~[doris-fe.jar:1.2-SNAPSHOT] ``` Introduced from #33610. When read meta image, the `resource` maybe null, we should ignore it.
``` Caused by: java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936) ~[?:1.8.0_301] at org.apache.doris.catalog.ResourceMgr.getResource(ResourceMgr.java:166) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.datasource.CatalogProperty.catalogResource(CatalogProperty.java:67) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.datasource.CatalogProperty.getOrDefault(CatalogProperty.java:77) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.datasource.ExternalCatalog.setDefaultPropsIfMissing(ExternalCatalog.java:173) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.datasource.hive.HMSExternalCatalog.setDefaultPropsIfMissing(HMSExternalCatalog.java:238) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.datasource.ExternalCatalog.gsonPostProcess(ExternalCatalog.java:687) ~[doris-fe.jar:1.2-SNAPSHOT] at org.apache.doris.persist.gson.GsonUtils$PostProcessTypeAdapterFactory$1.read(GsonUtils.java:640) ~[doris-fe.jar:1.2-SNAPSHOT] at com.google.gson.TypeAdapter.fromJsonTree(TypeAdapter.java:299) ~[gson-2.10.1.jar:?] at org.apache.doris.persist.gson.RuntimeTypeAdapterFactory$1.read(RuntimeTypeAdapterFactory.java:289) ~[doris-fe.jar:1.2-SNAPSHOT] ``` Introduced from #33610. When read meta image, the `resource` maybe null, we should ignore it.
Proposed changes
Introduced from #33610.
When read meta image, the
resource
maybe null, we should ignore it.Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...