-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[HIVE-28585]: Ensure SerDes is case insensitive for data type-'string' to align with HQL and SQL #5515
base: master
Are you sure you want to change the base?
Conversation
1238f9b
to
9260ab9
Compare
@check-spelling-bot Report🔴 Please reviewSee the files view or the action log for details. Unrecognized words (1)Coulmn Previously acknowledged words that are now absentaarry bytecode HIVEFETCHOUTPUTSERDE timestamplocal yyyyTo accept these unrecognized words as correct (and remove the previously acknowledged and now absent words), run the following commands... in a clone of the [email protected]:shivjha30/hive.git repository If the flagged items do not appear to be textIf items relate to a ...
|
|
|
||
| @Test | ||
| public void testColumnDeserializeCase() throws Exception { | ||
| props.setProperty(serdeConstants.LIST_COLUMN_TYPES, "STRING,STRING,STRING"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you modify the column type with different cases for the datatype
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the Review @Indhumathi27
Addressed the comment
…' to align with HQL and SQL
|
| @@ -384,7 +384,7 @@ public static PrimitiveTypeEntry getTypeEntryFromPrimitiveWritableClass( | |||
| * Get the TypeEntry for the given base type name (int, varchar, etc). | |||
| */ | |||
| public static PrimitiveTypeEntry getTypeEntryFromTypeName(String typeName) { | |||
| return typeNameToTypeEntry.get(typeName); | |||
| return typeNameToTypeEntry.get(typeName.toLowerCase()); | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The changes proposed in this pull request aim to make the SerDe API case-insensitive for the data type "string".
This change seems to affect all primitive type and not only "string" type.
Can this cause any problems?



What changes were proposed in this pull request?
The changes proposed in this pull request aim to make the SerDe API case-insensitive for the data type "string". Specifically, the modification involves adjusting the typeNameToTypeEntry map lookup to handle case insensitivity by converting the key to lowercase before performing the lookup.
Why are the changes needed?
The changes are needed to address inconsistencies between Hive and other connectors like Trino/Presto when using the SerDe API. Hive treats the data type "STRING" in a case-insensitive manner, but the current SerDe implementation does not. This discrepancy can lead to issues where SQL and HQL queries behave differently, causing potential data handling problems and integration issues. By ensuring case insensitivity in the SerDe API, we can maintain consistent behavior across different APIs and connectors interfacing with Hive.
This issue was observed in OpenCSVSerde
Does this PR introduce any user-facing change?
NO