xmlstarlet failed to load external entity

Question

My script is a simple one-liner that uses curl to get a URL with xml output. I'm trying to output only the text in the following tag:

<TEXT>No Hosts Queued for Purging</TEXT>

My script:

curl -u username:password -H 'X-Requested-With:QualysApiExplorer' 'https://qualysapi.qualys.com:443/api/2.0/fo/asset/host/' -d "action=purge&ips=$1&" | xmlstarlet sel -t -m '/BATCH_RETURN/RESPONSE/BATCH_LIST/BATCH/TEXT' -v "."

The curl output looks like this when not using xmlstarlet:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE BATCH_RETURN SYSTEM "https://qualysapi.qualys.com/api/2.0/batch_return.dtd">
<BATCH_RETURN>
  <RESPONSE>
    <DATETIME>2017-04-20T20:27:15Z</DATETIME>
    <BATCH_LIST>
      <BATCH>
        <CODE>1921</CODE>
        <TEXT>No Hosts Queued for Purging</TEXT>
      </BATCH>
    </BATCH_LIST>
  </RESPONSE>
</BATCH_RETURN>

I get the following error when using xmlstarlet:

failed to load external entity "https://qualysapi.qualys.com/api/2.0/batch_return.dtd"
TYPE BATCH_RETURN SYSTEM "https://qualysapi.qualys.com/api/2.0/batch_return.dtd"
                                                                               ^

Answer 1

You can use fo -D to remove the doctype before performing the query (sel) :

xml fo --help
XMLStarlet Toolkit: Format XML document
Usage: xml fo [<options>] <xml-file>
where <options> are
   -D or --dropdtd             - remove the DOCTYPE of the input docs

The following would work :

curl -u username:password \
     -H 'X-Requested-With:QualysApiExplorer' \
     'https://qualysapi.qualys.com:443/api/2.0/fo/asset/host/' \
     -d "action=purge&ips=$1&" | \
     xmlstarlet fo -D | \
     xmlstarlet sel -t -v "BATCH_RETURN/RESPONSE/BATCH_LIST/BATCH/TEXT"