Giganews header compression posted Tue, 25 May 2010 02:22:38 -0500
After messing around a bit with TCL, I finally figured out how to read the compressed headers from Giganews. Yay.
Thanks to a post over here, I was able to start with the basic NNTP conversation and add the rest I pieced together over the past couple of nights. My version with compression and SSL looks like this:
#!/usr/bin/tclsh
# load tls package
package require tls
# configure socket
set sock [tls::socket news.giganews.com 563]
fconfigure $sock -encoding binary -translation crlf -buffering none
# authenticate to GN
puts stderr [gets $sock]
puts stderr "sending user name"
puts $sock "authinfo user xxxxxxxx"
puts stderr [gets $sock]
puts stderr "sending password"
puts $sock "authinfo pass yyyyyyyy"
puts stderr [gets $sock]
# enable compression
puts stderr "sending xfeature command"
puts $sock "xfeature compress gzip"
puts stderr [gets $sock]
# set group
puts stderr "sending group command"
puts $sock "group giganews.announce"
puts stderr [gets $sock]
# issue xover command based on group posts
puts stderr "sending xover command"
puts $sock "xover 2-48"
set resp [gets $sock]
puts stderr $resp
# if the response is 224, parse the results
if {[lindex [split $resp] 0] == "224"} {
# loop through uncompressed results
# while {[gets $sock resp] > 0} {
# if {$resp == "."} {
# puts stdout $resp
# break
# }
# puts stdout $resp
# }
# loop through compressed results
while {[gets $sock resp] > 0} {
if {[string index $resp end] == "."} {
append buf [string range $resp 0 end-1]
break
}
append buf $resp
}
}
# uncompress those headers!
puts -nonewline stdout [zlib decompress $buf]
# issue a quit command
puts stderr "sending quit"
puts $sock quit
puts -nonewline stderr [read $sock]
Feel free to take the results and run. I'm not sure if there is a limit to how many headers you can fetch in a single go. I imagine it's more or less limited to your local buffer size, so don't grab too many at a time (at least in TCL I imagine). Anything more aggressive would require some fine tuning no doubt. But this was all just proof of concept to see if I could make it work. Now to write my newzbin.com replacement!