Discussion:
Over the top for .DBF?
(too old to reply)
badlange
2006-03-20 13:48:45 UTC
Permalink
Hi folks,

My web-application (CGI) for managing questionnaires has collected the
results of a big questionnaire (5000+ respondents, 300+ questions).

Now I have to export these results to a new DBF for use in SPSS, but ...
the export won't work. According to the docs a DBF can hold up to 1024
fields and 65535 bytes in record size but my 450-field-DBF can't be
created.

Can anyone help me?

grtx,
Ben de Lange
D.J.W. van Kooten
2006-03-20 18:04:10 UTC
Permalink
On Mon, 20 Mar 2006 08:48:45 -0500, "badlange"
Post by badlange
Now I have to export these results to a new DBF for use in SPSS, but ...
the export won't work. According to the docs a DBF can hold up to 1024
fields and 65535 bytes in record size but my 450-field-DBF can't be
created.
Can anyone help me?
Hello Ben,

You don't write what the original format is - is it a dbf which you
need to export to another dbf? The difference might be that we have
dbase III, dbase IV and Foxpro style dbf's. To see what SPSS expects,
read in the dbf in Excel and export it as the one of the 3 dbf's and
see what it takes. Realize the following:

Excel will use as type whatever it sees in the first field instead of
using the format. So enter a dummy field which you will with "X",
10.25, a date or whatever type you need. Besides, you still need to
format the columns.

Make sure the firsr row contains unique names up to 10 characters
without spaces. Those will be the field names.

Save as spreadsheet so you can modify again when next steps go wrong.

Select all and type Save As and save in the required type.

Accept the message that exporting might lose some info and don't save
again afterwards

I hope this helps.

Dick van Kooten
Ben
2006-03-20 22:07:56 UTC
Permalink
Dick,

The original format is also DBF (DBFCDX to be exact). This file holds a
record of four fields (respondent ID, question ID, option ID and free text
memo), this file needs to be reformatted to columns holding the question
ID's and records holding the respondent ID's.

SPSS can import DBFCDX directly, that's why I rather won't go any other way
than DBFCDX.

So, the only real question remains, why won't VO create my 450 field DBF?


B.
Post by D.J.W. van Kooten
On Mon, 20 Mar 2006 08:48:45 -0500, "badlange"
Post by badlange
Now I have to export these results to a new DBF for use in SPSS, but ...
the export won't work. According to the docs a DBF can hold up to 1024
fields and 65535 bytes in record size but my 450-field-DBF can't be
created.
Can anyone help me?
Hello Ben,
You don't write what the original format is - is it a dbf which you
need to export to another dbf? The difference might be that we have
dbase III, dbase IV and Foxpro style dbf's. To see what SPSS expects,
read in the dbf in Excel and export it as the one of the 3 dbf's and
Excel will use as type whatever it sees in the first field instead of
using the format. So enter a dummy field which you will with "X",
10.25, a date or whatever type you need. Besides, you still need to
format the columns.
Make sure the firsr row contains unique names up to 10 characters
without spaces. Those will be the field names.
Save as spreadsheet so you can modify again when next steps go wrong.
Select all and type Save As and save in the required type.
Accept the message that exporting might lose some info and don't save
again afterwards
I hope this helps.
Dick van Kooten
Geoff Schaller
2006-03-20 23:56:08 UTC
Permalink
Care to zip and email me the dbf/fpt?
I'm willing to take a look.
Post by Ben
Dick,
The original format is also DBF (DBFCDX to be exact). This file holds a
record of four fields (respondent ID, question ID, option ID and free text
memo), this file needs to be reformatted to columns holding the question
ID's and records holding the respondent ID's.
SPSS can import DBFCDX directly, that's why I rather won't go any other
way than DBFCDX.
So, the only real question remains, why won't VO create my 450 field DBF?
B.
badlange
2006-03-21 11:35:39 UTC
Permalink
Geoff,

Which DBF/FPT are you talking about? The one VO can't create (which I
don't have) or the source DBF (which is approx. 1.6 million records and 55
MB+).

B.
Geoff Schaller
2006-03-21 21:44:12 UTC
Permalink
The source one. How big is it zipped?

If its larger than 10MB then email me privately and I will send you a secure
FTP site to upload it to.

Geoff
Post by badlange
Geoff,
Which DBF/FPT are you talking about? The one VO can't create (which I
don't have) or the source DBF (which is approx. 1.6 million records and 55
MB+).
B.
Dave P
2006-03-21 22:27:52 UTC
Permalink
yahoo email supports 1.gig free now
so email should not be a issue
Post by Geoff Schaller
The source one. How big is it zipped?
If its larger than 10MB then email me privately and I will send you a secure
FTP site to upload it to.
Geoff
Post by badlange
Geoff,
Which DBF/FPT are you talking about? The one VO can't create (which I
don't have) or the source DBF (which is approx. 1.6 million records and 55
MB+).
B.
Gary Stark
2006-03-21 23:03:31 UTC
Permalink
Post by Dave P
yahoo email supports 1.gig free now
Gmail supports more than double that.

Email me if you want a gmail invitation. ..
--
g.
Gary Stark
***@RedbacksWeb.com
http://www.RedbacksWeb.com
Saulo Rodrigues
2006-03-22 00:30:19 UTC
Permalink
Hi

http://www.walla.com supports 3G mail account.

Cheers
Saulo
Geoff Schaller
2006-03-22 01:51:27 UTC
Permalink
I think you've all lost the plot <g>.

This has nothing to do with mail box storage but everything to do with mail
size. 10MB is a common limit but 2MB is often enforced by Administrators for
practical reasons. And emails over 10MB are a stupidity anyhow. FTP is the
simple solution. I'm just offering the guy some alternates. I don't see
where he asked for ISP alternates <g>.

Geoff
Post by Dave P
yahoo email supports 1.gig free now
so email should not be a issue
Post by Geoff Schaller
The source one. How big is it zipped?
If its larger than 10MB then email me privately and I will send you a
secure
Post by Geoff Schaller
FTP site to upload it to.
Geoff
Stephen Quinn
2006-03-21 01:13:54 UTC
Permalink
Ben
Post by Ben
So, the only real question remains, why won't VO create my 450 field DBF?
Maybe the macro compiler cannot handle a string length built using 450
fieldnames.

You don't say how your exporting the data, maybe you need to
DO WHILE !EOF()
// Write it out
SKIP()
ENDDO

Show us the relevant code (the DBF creation part as well)

HTH
Steve
Robert van der Hulst
2006-03-21 01:45:57 UTC
Permalink
Hi Ben,
On Mon, 20 Mar 2006, at 23:07:56 [GMT +0100] (which was 23:07 where I live)
you wrote about: 'Over the top for .DBF?'
Post by Ben
The original format is also DBF (DBFCDX to be exact). This file holds a
record of four fields (respondent ID, question ID, option ID and free text
memo), this file needs to be reformatted to columns holding the question
ID's and records holding the respondent ID's.
SPSS can import DBFCDX directly, that's why I rather won't go any other way
than DBFCDX.
So, the only real question remains, why won't VO create my 450 field DBF?
Send me the sample code and I will run it through the debugger to see
what causes your problem.
--
Robert van der Hulst
VO Development Team
www.sas-software.nl
Gary Stark
2006-03-20 22:26:42 UTC
Permalink
Ben,

When you say the the export doesn't work, what exactly do you mean?
What, exactly, is happenning?

First thoughts though are to look at potential data issues.
Post by badlange
Hi folks,
My web-application (CGI) for managing questionnaires has collected the
results of a big questionnaire (5000+ respondents, 300+ questions).
Now I have to export these results to a new DBF for use in SPSS, but ...
the export won't work. According to the docs a DBF can hold up to 1024
fields and 65535 bytes in record size but my 450-field-DBF can't be
created.
Can anyone help me?
grtx,
Ben de Lange
--
g.
Gary Stark
***@RedbacksWeb.com
http://www.RedbacksWeb.com
badlange
2006-03-21 11:50:27 UTC
Permalink
Gary,

After reading my message again, I saw that I didn't mention the reason.

The export won't work because VO won't create my 450-field-DBF (the docs
say a DBF can hold up to 1024 fields).

I'm creating an array with 450 fields in it and send it to the
DBFileSpec:Create() method. This array has no duplicate fieldnames in it.
Now the Create() method fails without any error-message.

If I limit the export-file to approx. 360 fields, it works fine (but then
I end up with only half of the results).

B.
Gary Stark
2006-03-21 11:54:23 UTC
Permalink
Post by badlange
Gary,
After reading my message again, I saw that I didn't mention the reason.
The export won't work because VO won't create my 450-field-DBF (the docs
say a DBF can hold up to 1024 fields).
I'm creating an array with 450 fields in it and send it to the
DBFileSpec:Create() method. This array has no duplicate fieldnames in it.
Now the Create() method fails without any error-message.
If I limit the export-file to approx. 360 fields, it works fine (but then
I end up with only half of the results).
B.
Have you tried loading the structure into a dbf file, and using the
CREATE FROM syntax?

Thos sounds like an issue with the way that the create table function is
being processed, but we'd need to see some code.
--
g.
Gary Stark
***@RedbacksWeb.com
http://www.RedbacksWeb.com
badlange
2006-03-21 13:24:22 UTC
Permalink
Hi folks,

When constructing some sample code for Robert van der Hulst I was able to
create a DBF containing 600+ fields. This convinced me that there was
still something wrong with my code.
And it turned out to be a wrong fieldtype ('S' instead of 'C') for one of
the fields.

This brings up the following question: why doesn't the DBFileSpec-class
raise an error when stumbling on this???

grtx,
Ben
Dave P
2006-03-21 15:48:48 UTC
Permalink
we are responsible for some things....
you could subclass the filespec or dbserver or
dbcreate function
put in your own error checking code....

if it cant create..than solve it a Development time

Dave P
Post by badlange
Hi folks,
When constructing some sample code for Robert van der Hulst I was able to
create a DBF containing 600+ fields. This convinced me that there was
still something wrong with my code.
And it turned out to be a wrong fieldtype ('S' instead of 'C') for one of
the fields.
This brings up the following question: why doesn't the DBFileSpec-class
raise an error when stumbling on this???
grtx,
Ben
Geoff Schaller
2006-03-21 21:49:57 UTC
Permalink
Ask Robert <g>.

Again, send him a sample of the code that doesn't work and I am sure he will
put it into his bag of tricks to fix.

Geoff
Post by badlange
Hi folks,
When constructing some sample code for Robert van der Hulst I was able to
create a DBF containing 600+ fields. This convinced me that there was
still something wrong with my code.
And it turned out to be a wrong fieldtype ('S' instead of 'C') for one of
the fields.
This brings up the following question: why doesn't the DBFileSpec-class
raise an error when stumbling on this???
grtx,
Ben
Dave P
2006-03-21 15:45:57 UTC
Permalink
I just threw this together and i was able too create a 800 field file
Its not your field definitions but numerics, logic,date and char values and
memo fields
VO27b 2740
Was able too create a big dbf....Open and browse
I use dbcreate in a function, I do not use filespec:create()

Dave P


FUNCTION dvtest()
LOCAL THEfIELDS AS ARRAY
LOCAL x AS INT
local nfcnt :=1023 as int
TheFields:=ArrayNew(nfcnt)
FErase("D:\VODEV\TEST\DAVE.DBF" )
FOR x:=1 TO nfcnt
TheFields[x]:={"_"+NTrim(x),"C",100,0}
IF X==15 // PUT A MEMO IN THER
TheFields[x]:={"_"+NTrim(x),"M",10,0}
LOOP
ENDIF
IF x % 3==0
TheFields[x]:={"_"+NTrim(x),"N",9,2}
ENDIF
IF x % 10==0
TheFields[x]:={"_"+NTrim(x),"L",1,0}
ENDIF
IF X % 30 ==0
TheFields[x]:={"_"+NTrim(x),"D",8,0}
ENDIF
NEXT
dvCREATEDBF("D:\VODEV\TEST\DAVE.DBF",TheFields)






FUNCTION dvCREATEDBF(uFilename AS USUAL,uStru AS USUAL) AS LOGIC
// MUST EMBED OPTIONAL PATH IN FILE NAME
//IF CURRENT PATH IS NOT WHERE U WANT THE FILE CREATED
LOCAL lRet:=TRUE AS LOGIC
LOCAL aStru AS ARRAY
LOCAL oFs AS FileSpec
IF IsObject(uFilename) // filespec
ofs:=uFilename
ELSE
ofs:=Filespec{uFileName}
ENDIF
IF IsObject(uStru)
aStru:=uStru:DBSTRUCT
ELSE
aStru:=ustru
ENDIF
uFilename:=ofs:FullPath

FErase(String2Psz(uFileName))
IF (lRet:=DBCREATE(uFilename,aStru,,TRUE,"Z_AAAAAA",,FALSE))
DBSELECTAREA("Z_AAAAAA")
DBCLOSEAREA()
ENDIF
RETURN(lRet)

Dave
Post by badlange
Hi folks,
My web-application (CGI) for managing questionnaires has collected the
results of a big questionnaire (5000+ respondents, 300+ questions).
Now I have to export these results to a new DBF for use in SPSS, but ...
the export won't work. According to the docs a DBF can hold up to 1024
fields and 65535 bytes in record size but my 450-field-DBF can't be
created.
Can anyone help me?
grtx,
Ben de Lange
Continue reading on narkive:
Loading...