Linux Help
guides forums blogs
Home Desktops Distributions ISO Images Logos Newbies Reviews Software Support & Resources Linuxhelp Wiki

Welcome Guest ( Log In | Register )



Advanced DNS Management
New ZoneEdit. New Managment.

FREE DNS Is Back

Sign Up Now
 
Reply to this topicStart new topic
> Scripting help, Removing everything from a log file before a character
jennyjebson
post Dec 3 2007, 06:49 AM
Post #1


Whats this Lie-nix Thing?
*

Group: Members
Posts: 2
Joined: 30-November 07
Member No.: 13,168



Hello,

I am a newbie, and need a little assistance!

I have an application that is running on Linux that produces a log file.

The information in the log file is produced by purely enabling the debug option in the application, however there is far too much

non useful information in the log files produced.

I would like to cut out some of the information in the log file... there seems to be a unique character after which I think is useful information.

So I would like to remove all characters before this unique character on each line "|" and output this to a new log file.

Firstly, is this possible? as anyone able to help me make a little script?

Here is an excerpt from the the log file produced:

30-11 11:55:06 expect: Timeout(30) Max Len(60) EndLen(2) FailFlag(0) StartLen(0)
30-11 11:55:06 toggle_HUP: Hangup line
30-11 11:55:08 sending command:
0D 0A | ..

30-11 11:55:11 expect: Timeout Occured

30-11 11:55:11 main: Failed - Next Command = 10
30-11 11:55:11 expect: Timeout(10) Max Len(50) EndLen(0) FailFlag(0) StartLen(0)
30-11 11:55:14 sending command:
0D 0A 0D 0A | ....

30-11 11:55:15 expect: Timeout Occured

30-11 11:55:15 main: Success - Next Command = 50
30-11 11:55:15 expect: Timeout(40) Max Len(1600) EndLen(5) FailFlag(0) StartLen(4)
30-11 11:55:16 sending command:
0D 40 30 31 20 68 20 31 0D | .@01 h 1.

30-11 11:55:16 expect: Found Start String
30-11 11:55:16 expect Found Reply String (position 357) (minLen 0)
30 31 0D 0A 23 0D 0A 0D 0A 32 30 30 37 2D 31 31 | 01..#....2007-11
2D 32 38 20 20 32 33 3A 35 30 2C 30 31 2C 48 31 | -28 23:50,01,H1
34 2C 52 4F 53 41 0D 0D 0A 30 31 20 20 20 39 2E | 4,ROSA...01 9.
39 3B 30 32 20 20 20 20 37 37 3B 30 33 20 20 20 | 9;02 77;03

Is anyone able to help me?

Thanks,

jenny
Go to the top of the page
 
+Quote Post
michaelk
post Dec 3 2007, 04:06 PM
Post #2


Its GNU/Linuxhelp.net
*******

Group: Support Specialist
Posts: 1,800
Joined: 23-January 03
Member No.: 360



So what is this application? What are you trying to accomplish? IMHO the entire log file has useful information.

What is you see is the hex number that corresponds to the ASCII character displayed on the right side of the |. The | is not real data and it is referred to as a pipe
pipe character in linux speak. This format is typical of hex file viewers/editors. If there isn't a readable character for the hex code i.e. [A-Z, a-z, 0-9] etc then a "."
is used. A 0D 0A is a carriage return, line feed sequence.

30 31 0D 0A 23 0D 0A 0D 0A 32 30 30 37 2D 31 31 | 01..#....2007-11
2D 32 38 20 20 32 33 3A 35 30 2C 30 31 2C 48 31 | -28 23:50,01,H1
34 2C 52 4F 53 41 0D 0D 0A 30 31 20 20 20 39 2E | 4,ROSA...01 9.
39 3B 30 32 20 20 20 20 37 37 3B 30 33 20 20 20 | 9;02 77;03

What you would see if viewing from a text editor, vi, kedit, notepad etc is:
01
#

2007-11-28 23:50,01,H14,ROSA
01___9.9;02 77;03___

I've used a "_" to display spaces i.e the hex character 20. If you want to know more about ACSII text characters you can find lots of info by googling.
From what I have gathered from the log file this is a response string from the transmitted command. What you want is possible but might remove
raw information that could be important.
Go to the top of the page
 
+Quote Post
jennyjebson
post Dec 5 2007, 07:16 AM
Post #3


Whats this Lie-nix Thing?
*

Group: Members
Posts: 2
Joined: 30-November 07
Member No.: 13,168



QUOTE (michaelk @ Dec 3 2007, 04:06 PM) *
So what is this application? What are you trying to accomplish? IMHO the entire log file has useful information.

What is you see is the hex number that corresponds to the ASCII character displayed on the right side of the |. The | is not real data and it is referred to as a pipe
pipe character in linux speak. This format is typical of hex file viewers/editors. If there isn't a readable character for the hex code i.e. [A-Z, a-z, 0-9] etc then a "."
is used. A 0D 0A is a carriage return, line feed sequence.

30 31 0D 0A 23 0D 0A 0D 0A 32 30 30 37 2D 31 31 | 01..#....2007-11
2D 32 38 20 20 32 33 3A 35 30 2C 30 31 2C 48 31 | -28 23:50,01,H1
34 2C 52 4F 53 41 0D 0D 0A 30 31 20 20 20 39 2E | 4,ROSA...01 9.
39 3B 30 32 20 20 20 20 37 37 3B 30 33 20 20 20 | 9;02 77;03

What you would see if viewing from a text editor, vi, kedit, notepad etc is:
01
#

2007-11-28 23:50,01,H14,ROSA
01___9.9;02 77;03___

I've used a "_" to display spaces i.e the hex character 20. If you want to know more about ACSII text characters you can find lots of info by googling.
From what I have gathered from the log file this is a response string from the transmitted command. What you want is possible but might remove
raw information that could be important.


Hello,

The audit is provided by an in-house application.

As you say the rest of the information in the log file is useful - but unfortunately not to our technical support team who would basically want the audit to look like:

| ..
| ..
| ....
| .@01 h 1.
| 01..#....2007-11 (etc)
| @01 TIME 11 55 1

Its basically a log of information on contact to a list of modems connected to outstations.
Go to the top of the page
 
+Quote Post
markjr
post Jan 15 2008, 10:59 PM
Post #4


./configure
***

Group: Admin
Posts: 62
Joined: 9-February 06
Member No.: 6,054



From a command line I would do this:

CODE
grep \| <logfile> | cut -d\| -f2


The "cut" command gives you access to a given field -f separated by delimiter -d, your original post said <i>before</i> the | but the rest of the thread it seemed like you wanted everything after the |, hence the -f2 but if you really do want before, just use -f1

So this wouldn't happen in realtime but maybe you could redirect to a new file, run out of crontab or something


CODE
grep \| <logfile> | cut -d\| -f2 > <newlog>
Go to the top of the page
 
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 19th November 2017 - 10:56 AM