Thursday, June 20, 2013

Mapping Source-to-Target analyzer

Below is the original post decribing the early version and the main feature of the tool. For up to date description please visit XML Analyzer for Informatica Powercenter blog post.



Do these questions sound familiar?
  • "Hey, can you tell me where does this value come from?" 
  • "Do you know where does data for this column come from?"
  • "If I change values here, what columns will be affected?"
One thing great about PowerCenter is, it's very easy to trace column dependancies. You can quckly choose "Select Link Path" on desired column and trace it forward, backward or both, e.g:


Looks good. But what if you get more of those questions, like:
  • "Can you send me an email describing columns A, B, C, D, F?"
  • "Can you quickly send me some simple table with column dependancies?"
Select Link Path is extremly useful for development, but it is simply not possible to use it like 200 times if you need to create some document for large table. And what if there are more sources and targets?
One way is to use Metadata Manager. But what if it is not available? If you'd need something quick and handy, you can use:

Online Mapping XML Analyzer

It's a simple tool allowing you to create a Source to Target dependancy description in (almost) three clicks:
  1. Open the explorer
  2. Choose the mapping XML file
  3. Click "Upload"
As a result, you'll get a table as shown below:
In this example you can see all source columns listed (with column name, name of the source and source type) on the left with the corresponding target columns on the right. In addition you can see BusinessDate target column that is not being populated by any source column. It comes from an expression exp_Convert_Datatypes mentioned on the left. 

This way you can see:
  • all source columns and the target they affect (many to many relations supported),
  • all target columns with corresponding source (even if it is not being populated by source).
In addition, you can copy the whole table to Excel and do whatever you need to make it suit your needs (format, add filters, etc.):




Please keep in mind that it is a work-in-progress. Feel free to let me know if you face issues, but be advised that I cannot guarantee to fix them and I also hold no responsibility for using the tool.

I hope this will be helpful!

31 comments:

  1. Great!!!

    Is it possible add in table the columns datatypes of source/target?

    ReplyDelete
    Replies
    1. I will look into it. Thing is, the tables would grow. And when the mapping is large (and/or there are many mappings in XML file), the data flow analysis is quite resource consuming. Online app quota is very limited and I need to keep that in mind.
      There would be no problem however in extending the service version available on Informatica Marketplace: https://community.informatica.com/solutions/xml_analyzer

      Delete
    2. I'm happy to say that I've been able to extend the functionality a lot - datatypes for sources and targets are now also included! Give it a try at powercenter-xmlanalyzer.appspot.com !

      Delete
  2. Can we get Mapping Name in the table ..? Is there a offline code..?

    ReplyDelete
    Replies
    1. 1. Adding Mapping Name is possible. It would make the table wider, which I'm trying to avoid. What if I'd include the Mapping Name just in the downloadable CSV file? Please share your thoughts on this.

      2. All the available versions are described in separate post.

      3. A demo version is available on Informatica Marketplace.

      Delete
    2. I've added a column with the Mapping Name as part of the downloadable csv file. Let me know if it works for you.

      Delete
    3. I have been using this online tool and it is great. However I have 1 mapping that does not generate HTML.
      Are there size limitations on the XML file.

      Delete
    4. I'm so glad to hear you like it!

      Please provide some more information. At least the file size. Would it be possible to put the file somewhere I could download and try to identify the issue?
      The online version serves mostly the demo purpose - the resources of free AppEngine are quite strictly limited. There are no such limitation in the full version which can be found here.

      Thank you very much for your comment here! I'll be grateful if you'd spread the word so others can also try the tool :)

      Delete
  3. I could send it to you in an email, 213 Kb

    ReplyDelete
    Replies
    1. Please send it to infa.scripting - at gmail. I'll see what might be the issue (it doesn't seem to be the size of the file) and get back to you.

      Delete
  4. The app is working great for simple mappings. But when I use a mapping that has an xml of 10 MB or more, the app is not working at all. I tried downloading the offline version but there are not proper instructions of how to install or use the app. Please help. We are doing an impact analysis and this tool can greatly help if it works.

    Thanks!
    DM

    ReplyDelete
  5. Hi Td,

    Great Job. I was looking for it and trying it through the informatica repository tables.
    Tell me one thing will it still give the correct linking if there is a mapplet in between.

    Regards,
    Udayan

    ReplyDelete
    Replies
    1. Glad to see another satisfied user! Mapplet support has been recently added. Give it a try, verify - and let me know in case of any issues.

      Delete
  6. Td,

    One more question i have for you. If i have a single XML file for muliple mappings will it give me the same result as mentioned above in the post.

    regards,
    Udayan

    ReplyDelete
    Replies
    1. Multiple objects in single XML file work fine as well. Just keep one thing in mind: the online free version is hosted on appspot with limited resources and it might not get you the result of the analysis because of that. Just let me know if you'd need any help or would just have some questions.

      Delete
  7. Hi td,
    Great work !! interested to know how you built it. Can you please share some insights? Which language or something to start with..

    ReplyDelete
    Replies
    1. Hi Kavinaesh!

      I'm really glad you like it! Keeping in mind that the online tool has very limited resources, please treat is as a demo. Let me know if you'd be interested in obtaining a full, stand-alone version that can be deployed within your organization.

      Both the full version as well as this online demo tool has been written in Python. It required a bit of programming skills and a very deep understanding of Informatica insights.

      Delete
    2. Hi td,
      Thanks for the response. will be in touch.

      Delete
    3. Hi Guys,
      I have built the same kind of tool using java. XML DomParser and NodeList was helpful for me. I had around 300 mappings needed to be analyzed and used to get regular requirements to present column to column mappings in excel. Most of my mappings were pass through so I didnt make the tool complex. Would like to know about the your implementation process.

      Cheers,
      Rishi

      Delete
    4. Hi Rishinder,

      this one is created with Python and works server-side. It's designed to deal with complete workflows, including complex mappings and mapplets.
      It's not perfect though - there is a plan to rebuild it and also update the ugly presentation layer ;)

      Delete
  8. This tool is good for code review prospective. However I tried to evaluate Source to Target mapping logic which provides the data lineage. It is working for simple mapping but it is not showing accurate results for big mappings I mean if mapping has 4 expression sand 4 lookups and 3 aggregators. It is not showing exact source to target mapping logic. Can you please help on this?

    ReplyDelete
    Replies
    1. Hi! Thanks for the comment! The tool has been tested with some sophisticated mappings - but I can imagine some things I have not forseen. Would you be able to contact me on infa.scripting@gmail.com and provide more details?

      Delete
  9. I have downloaded the off-line version. When I run on a mapping xml extract, I get this error: "error: global name 'SESSTRANSFORMATIONINST' is not defined".
    When I run against a workflow xml, I get an html report, but it does not contain Source-to-Target dependency analysis, which is my main goal !
    Did I do something wrong ?
    thanks, and congrats for the development
    Jean-Paul

    ReplyDelete
    Replies
    1. Hi! Would you be able to contact me on infa.scripting@gmail.com and provide your xml file? Have you been trying it on powercenter-xmlanalyzer.appspot.com? What's the result? Which tool did you use to export: Workflow or Repository Manager? If the first, try the latter.

      Delete
  10. Too bad workflows are not really working smoothly...

    ReplyDelete
    Replies
    1. Can you please provide some more details? Perhaps I'd be able to help?
      For the online version please keep in mind that it's a demo with very limited resources. Let me know on infa.scripting@gmail.com if you'd be interested in full version.

      Delete
  11. After downloading the zip file , i have unzipped the content, Please let me know on how to run the code

    ReplyDelete
  12. Thank you for sharing an inspiring blog for Informatica learners, keep share more content on Informatica Online Training

    ReplyDelete
  13. I have been using this tool for quite some time now and its really a great one.
    I forgot my password for the same and reset password is not working.Can you help me with that ?

    ReplyDelete
    Replies
    1. I'll try to help. Please contact me via infa.scripting@gmail.com

      Delete