Comparing two text files with non-adjacent lines for unique

I am trying to find an easy and fast way to compare two files, each
with several thousand lines - only one column and spit out what is
unique only to one of the files.
So, compare file A and file B, and only lines that re unique to file A
are spit out to a new file.. comm and diff / sort and Uniq do not
work because in this case the two files will have non-adjacent lines.

Any help is GREATLY appreciated. Thank you in advance!
-TT

<tntelle@yahoo.com> wrote in message news:1187142111.587829.251580@q3g2000prf.googlegro ups.com...

I am trying to find an easy and fast way to compare two files, each
with several thousand lines - only one column and spit out what is
unique only to one of the files.
So, compare file A and file B, and only lines that re unique to file A
are spit out to a new file.. comm and diff / sort and Uniq do not
work because in this case the two files will have non-adjacent lines.

cat A A B | sort |uniq -u
awk 'FNR==NR{Seen[$0]++} FNR!=NR && !Seen[$0]' A B

I am not sure what you mean by "non-adjacent lines".
And note that the two solutions above give different results for
lines that appear more than once in B: it is not clear what you want.
Surely solutions based on diff or comm will work if you first sort
A and B?

--
John.

On 2007-08-15, tntelle@yahoo.com wrote:

I am trying to find an easy and fast way to compare two files, each
with several thousand lines - only one column and spit out what is
unique only to one of the files.
So, compare file A and file B, and only lines that re unique to file A
are spit out to a new file.. comm and diff / sort and Uniq do not
work because in this case the two files will have non-adjacent lines.

grep -Fvf FileB FileA

--
Chris F.A. Johnson, author <http://cfaj.freeshell.org/shell/>
Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
===== My code in this post, if any, assumes the POSIX locale
===== and is released under the GNU General Public Licence

John L wrote:

tntelle@yahoo.com> wrote in message news:1187142111.587829.251580@q3g2000prf.googlegro ups.com...
I am trying to find an easy and fast way to compare two files, each
with several thousand lines - only one column and spit out what is
unique only to one of the files.
So, compare file A and file B, and only lines that re unique to file A
are spit out to a new file.. comm and diff / sort and Uniq do not
work because in this case the two files will have non-adjacent lines.


cat A A B | sort |uniq -u
awk 'FNR==NR{Seen[$0]++} FNR!=NR && !Seen[$0]' A B

I am not sure what you mean by "non-adjacent lines".
And note that the two solutions above give different results for
lines that appear more than once in B: it is not clear what you want.
Surely solutions based on diff or comm will work if you first sort
A and B?

--
John.

Since he wants the lines that are in A but not in B,
I think the order of the files should be reversed.

awk 'NR==FNR{seen[$0]++; next} !seen[$0]' B A

Another way:

awk 'NR==FNR{seen[$0]; next} !($0 in seen)' B A

One Ruby solution:

ruby -e 'def lines;gets(nil).split("\n") end; puts lines - lines' A B

On Aug 15, 2:37 am, William James <w_a_x_...@yahoo.com> wrote:

John L wrote:
tnte...@yahoo.com> wrote in messagenews:1187142111.587829.251580@q3g2000prf.go oglegroups.com...
I am trying to find an easy and fast way to compare two files, each
with several thousand lines - only one column and spit out what is
unique only to one of the files.
So, compare file A and file B, and only lines that re unique to file A
are spit out to a new file.. comm and diff / sort and Uniq do not
work because in this case the two files will have non-adjacent lines.

cat A A B | sort |uniq -u
awk 'FNR==NR{Seen[$0]++} FNR!=NR && !Seen[$0]' A B

I am not sure what you mean by "non-adjacent lines".
And note that the two solutions above give different results for
lines that appear more than once in B: it is not clear what you want.
Surely solutions based on diff or comm will work if you first sort
A and B?

--
John.

Since he wants the lines that are in A but not in B,
I think the order of the files should be reversed.

awk 'NR==FNR{seen[$0]++; next} !seen[$0]' B A

Another way:

awk 'NR==FNR{seen[$0]; next} !($0 in seen)' B A

One Ruby solution:

ruby -e 'def lines;gets(nil).split("\n") end; puts lines - lines' A B- Hide quoted text -

- Show quoted text -

Thank you all ---
When i attempt awk 'NR==FNR{seen[$0]++; next} !seen[$0]' B A - i get
awk: syntax error near line 1
awk: bailing out near line 1

=/

On Wed, 15 Aug 2007 13:53:32 -0000, tntelle@yahoo.com
<tntelle@yahoo.com> wrote:

Thank you all ---
When i attempt awk 'NR==FNR{seen[$0]++; next} !seen[$0]' B A - i get
awk: syntax error near line 1
awk: bailing out near line 1

You are probably using Solaris Old Broken Awk(tm). Use nawk or

/usr/xpg4/bin/awk.

--
BOFH excuse #331:

those damn raccoons!

On Aug 15, 10:05 am, Bill Marcum <marcumb...@bellsouth.net> wrote:

On Wed, 15 Aug 2007 13:53:32 -0000, tnte...@yahoo.com <tnte...@yahoo.com> wrote:

Thank you all ---
When i attempt awk 'NR==FNR{seen[$0]++; next} !seen[$0]' B A - i get
awk: syntax error near line 1
awk: bailing out near line 1

You are probably using Solaris Old Broken Awk(tm). Use nawk or
/usr/xpg4/bin/awk.

--
BOFH excuse #331:

those damn raccoons!

haha
-- okay, got that down BUT now.... (more raccoons)

bash-2.03$ /usr/xpg4/bin/awk 'NR==FNR{seen[$0]++; next} !seen[$0]' ./
all.ldap.users ./all.oracle.users
input file "./all.ldap.users"input file "./all.oracle.users"bash-2.03$

it just says that, doesnt do anything more
= *(

On Aug 15, 10:23 am, tnte...@yahoo.com wrote:

On Aug 15, 10:05 am, Bill Marcum <marcumb...@bellsouth.net> wrote:

On Wed, 15 Aug 2007 13:53:32 -0000, tnte...@yahoo.com <tnte...@yahoo.com> wrote:

Thank you all ---
When i attempt awk 'NR==FNR{seen[$0]++; next} !seen[$0]' B A - i get
awk: syntax error near line 1
awk: bailing out near line 1

You are probably using Solaris Old Broken Awk(tm). Use nawk or
/usr/xpg4/bin/awk.

--
BOFH excuse #331:

those damn raccoons!

haha
-- okay, got that down BUT now.... (more raccoons)

bash-2.03$ /usr/xpg4/bin/awk 'NR==FNR{seen[$0]++; next} !seen[$0]' ./
all.ldap.users ./all.oracle.users
input file "./all.ldap.users"input file "./all.oracle.users"bash-2.03$

it just says that, doesnt do anything more
= *(
..... anybody!? I'll give you a cookie...

On Thu, 16 Aug 2007 06:06:49 -0700, tntelle@yahoo.com
<tntelle@yahoo.com> wrote:

On Aug 15, 10:23 am, tnte...@yahoo.com wrote:
On Aug 15, 10:05 am, Bill Marcum <marcumb...@bellsouth.net> wrote:

On Wed, 15 Aug 2007 13:53:32 -0000, tnte...@yahoo.com <tnte...@yahoo.com> wrote:

Thank you all ---
When i attempt awk 'NR==FNR{seen[$0]++; next} !seen[$0]' B A - i get
awk: syntax error near line 1
awk: bailing out near line 1

You are probably using Solaris Old Broken Awk(tm). Use nawk or
/usr/xpg4/bin/awk.

--
BOFH excuse #331:

those damn raccoons!

haha
-- okay, got that down BUT now.... (more raccoons)

bash-2.03$ /usr/xpg4/bin/awk 'NR==FNR{seen[$0]++; next} !seen[$0]' ./
all.ldap.users ./all.oracle.users
input file "./all.ldap.users"input file "./all.oracle.users"bash-2.03$

it just says that, doesnt do anything more
= *(
.... anybody!? I'll give you a cookie...

The error message looks like it might have something to do with Windows

end-of-line characters.



--
Function reject.

On Aug 16, 10:04 am, Bill Marcum <marcumb...@bellsouth.net> wrote:

On Thu, 16 Aug 2007 06:06:49 -0700, tnte...@yahoo.com



tnte...@yahoo.com> wrote:

On Aug 15, 10:23 am, tnte...@yahoo.com wrote:
On Aug 15, 10:05 am, Bill Marcum <marcumb...@bellsouth.net> wrote:

On Wed, 15 Aug 2007 13:53:32 -0000, tnte...@yahoo.com <tnte...@yahoo.com> wrote:

Thank you all ---
When i attempt awk 'NR==FNR{seen[$0]++; next} !seen[$0]' B A - i get
awk: syntax error near line 1
awk: bailing out near line 1

You are probably using Solaris Old Broken Awk(tm). Use nawk or
/usr/xpg4/bin/awk.

--
BOFH excuse #331:

those damn raccoons!

haha
-- okay, got that down BUT now.... (more raccoons)

bash-2.03$ /usr/xpg4/bin/awk 'NR==FNR{seen[$0]++; next} !seen[$0]' ./
all.ldap.users ./all.oracle.users
input file "./all.ldap.users"input file "./all.oracle.users"bash-2.03$

it just says that, doesnt do anything more
= *(
.... anybody!? I'll give you a cookie...

The error message looks like it might have something to do with Windows
end-of-line characters.

--
Function reject.- Hide quoted text -

- Show quoted text -

This is on a solaris system,...