You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Initially posted in Slack but w/o any traction. This question is two-fold:
Is there a way to use the LivyOperator with a proxy? My first attempt at using the operator with a custom livy_default connection resulted in max retries exceeded errors. Then I noticed LivyHook was inheriting from HttpHook, so I tried a simple GET request to the same endpoint using SimpleHttpOperator and it failed with the same error. I then re-tried SimpleHttpOperator with proxies passed in and it worked. The problem is that the extra_options argument is not available in the LivyOperator; is there any other way to pass proxy information to the operator?
Assuming there is no way to pass proxy info to the LivyOperator through code, is there any way to add proxy info to an HTTP Airflow Connection object itself? (Perhaps through the "extras" option in the UI?).
I am using Airflow 1.10.12 and the below two packages to interact with Livy:
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
Initially posted in Slack but w/o any traction. This question is two-fold:
LivyOperatorwith a proxy? My first attempt at using the operator with a customlivy_defaultconnection resulted in max retries exceeded errors. Then I noticedLivyHookwas inheriting fromHttpHook, so I tried a simple GET request to the same endpoint usingSimpleHttpOperatorand it failed with the same error. I then re-triedSimpleHttpOperatorwith proxies passed in and it worked. The problem is that theextra_optionsargument is not available in theLivyOperator; is there any other way to pass proxy information to the operator?LivyOperatorthrough code, is there any way to add proxy info to an HTTP Airflow Connection object itself? (Perhaps through the "extras" option in the UI?).I am using Airflow
1.10.12and the below two packages to interact with Livy:Beta Was this translation helpful? Give feedback.
All reactions