TO_DATE or not TO_DATE

Like a middle aged man visiting the gym for the first time in 10 years, I am merely going to flex my blogging muscles here…

One of the recurring issues that annoys me on the OTN SQL & PL/SQL forum, and also the APEX forum is the misunderstanding of what the TO_DATE function actually does. I think many people that are new to database development and Oracle see TO_DATE and it’s format mask and think – great I can reformat a date!

Well, no. That’s not what it does and is not what its for. Let’s get back to basics.

A date is a date. It’s not ’14-03-2013′, ’03-14-2013′, ’14 Mar 2013′ or any other representation you can think of. It’s a date, and in Oracle’s world, a time too. The moment that you are reading this is a date.

A string (or char or literal or whatever you want to call it – this post will use string from now on) is a series of characters. ‘Ben’ is a string, in the same way that ’14 March 2013′ is a string, they mean very little to the database unless they’re relating to something else.

So, to the purpose of TO_DATE()!

TO_DATE() is a system function that allows you to describe to the database a date that you are passing in as a string. The conversation would go something like this…

Developer: Yo DB! Got an input here, it’s ’14-MAR-2013′ – knock yourself out.

DB: Cool! I like strings.

Developer: But it’s a date?

DB: Oh. You didn’t mention that. What date is it?

Developer: I have to explain everything? You just can’t get the staff these days! It’s in DD-MON-YYYY format obviously…

DB: You should have just said, no need to be abusive.

In code, that would be TO_DATE(’14-MAR-2013′,’DD-MON-YYYY’). The database now knows it’s a date and can treat it as such. The Oracle doc’s and a million blog posts will tell you the many forms in which you can ‘describe’ the date to the database. So, that is TO_DATE(), and that’s what it does.

So why do people get confused? Well, mostly because they see the term ‘format mask’ and they’ve heard of TO_CHAR(). Then think the two functions can be used interchangeably because they both have ‘format masks’.

TO_CHAR() is a system function that allows you to describe to the database how to return something as a string. The conversation would go something like this…

Developer: Yo DB! Got a date here, just pop it back to me as a string so the user can see it please.

DB: Cool! Here you go – ’04-03-2013′

Developer: The user’s American?

DB: Oh, you didn’t mention that. How do they want it?

Developer: I have to explain everything? You just can’t get the staff these days! It’s in MM-DD-YYYY format obviously…

DB: You should have just said, no need to be abusive.

In code, that would be TO_CHAR(the_date,’MM-DD-YYYY’).

By now you probably wondering why the Developer keeps saying ‘Yo’ and the DB keeps replying with ‘Cool!’. That there is implicit conversion. The DB *knows* that the developer is a 1980’s pop rap singer, so it knows how to respond. This is all in your NLS parameters, there you can set the default date format for the database. This is why many people come across the situation of “this always works but suddenly the results are wrong!” when dealing with a predicate like…

my_date_column = :p_myDate

Now, let’s say the NLS parameters are set to ‘DD-MM-YYYY’. All the users input dates in that format. Then they employ an American and they input a date as ’03-04-2013′. The DB thinks that this is 3rd April 2013, but the user meant 4th March 2013.

You should never rely on implicit conversion (the same applies to numbers here), you should ALWAYS describe to the database explicitly what you are giving it.

Note to APEX developer’s here – every item that you have is returned as a string, so all of your reports etc. that refer to items as bind variables should be explicitly converted if they are supposed to be numbers or dates – saying “but it works at the moment” is not good enough!. APEX does do some clever things around this but I would always convert explicitly in my code.

Why does this all annoy me so much? Well, I see people doing nested TO_CHAR(TO_DATE(TO_CHAR())) all the time in the forums and wondering why their code isn’t working. Just remember – TO_DATE() describes to the database that you are giving it a date, TO_CHAR() can allow you to return a date as a string in a specified format. Stop being abusive to your database!

Okay, so that’s the blogging hiatus finished – I’ll be back soon!

Handling aggregated strings in SQL reports

Well, it’s been a long time without a new post from me so I thought I’d get the ball rolling again!

Having been surfing about and happened upon Adrian Billington’s site again, I spotted something pretty nifty. With a bit of rewriting, I started to see a possible benefit to the APEX community – mostly for the handling of multiselect list values in reports.

A common approach to dealing with them is to concatenate colons to the multiselect list value returned in the bind variable and then concatenating colons to the column being searching and using INSTR. Whilst this is okay for small sets of data, it can become problematic for large ones because it will negate the index on the searched column.

Consider this example of the that approach…

SELECT *
FROMĀ  emp
WHERE INSTR(':'||:LIST_ITEM||':',':'||empno||':')>0

Here, If you have an index on empno, this won’t be used and a full table scan will occur.

Now consider this (and this is seriously quick)…

WITH t_str AS
(
 SELECT  :LIST_ITEM||':'                                              AS str,
 (LENGTH(:LIST_ITEM) - LENGTH(REPLACE(:LIST_ITEM,':'))) + 1 AS no_of_elements
 FROM    dual
),   t_n_rows AS
(
 SELECT  LEVEL AS i
 FROM    dual
 CONNECT BY LEVEL <= (SELECT SUM(no_of_elements) FROM t_str)
),   t_in_list AS
(
 SELECT SUBSTR(str,start_pos,(next_pos - start_pos)) AS element_value
 FROM   (
 SELECT t_str.str,
 nt.i AS element_no,
 INSTR(t_str.str,':',DECODE(nt.i,1,0,1),
 DECODE(nt.i,1,1,nt.i - 1)) + 1 AS start_pos,
 INSTR(t_str.str,':',1,DECODE(nt.i,1,1,nt.i))       AS next_pos
 FROM   t_str JOIN t_n_rows nt
 ON nt.i <= t_str.no_of_elements
 )
)
SELECT *
FROM   emp
WHERE  empno IN (SELECT TO_NUMBER(element_value) FROM t_in_list);

Voila! The index on empno will be used! I know there’s more lines of code here but the performance benefits of it are huge (most of it’s done just using fast dual in the factored subquery). All you need to do is save it in your toolbox and copy & paste when the need arises!

Hope you enjoyed this, more posts coming soon.

 

PS: Sorry – forgot to link to the actual article by Adrian – hit the word ‘nifty’ now!